-
-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot retrieve data from key value store agent #3349
Comments
UPDATE: With this event formatter agent it forwards all data from the key value store agent as a new event:
Now the CSV Agent receives an Event that looks like this:
HOWEVER the problem now is that the CSV Agent outputs this, regardless of the data_path I set it to:
It's impossible to read the separate values and the resulting CSV is useless. If I set data_path to $.*.title it gives the error: If I set data_path to $.* and Use field to price the output is:
This doesn't make sense. |
I never did figure out how to get the K/VS Agent to work. I do, however, recognize the problem you're seeing now: You're getting serialized JSON and not "real" JSON. There might be an in-Huginn fix for that but I'm using Jq Agent to deserialize the strings into JSON documents that can then be processed.
..should do what you need. The Liquid module as used by Huginn has a |
I want to have a CSV File at the end, which I can upload via ftp. I already managed to get an object/json out of K/VS if you read my second comment. |
I did read it. You have to de-serialize the JSON (all of the |
Where do I put that filter you suggest? The CSV Agent already receives an object (the description says it can read objects??). This is what the CSV Agent receives:
Update: When I use an JQ Agent and send the Events from the Event Formatter there it says this error:
|
Right after your Csv Agent. |
Results in this error:
Besides, what will the JQ Agent output? I need a comma separated CSV File, not a JSON. One of the problems is that I passed an object instead of json to the CSV Agent. But even when I change the Event Formatting Agent to output proper json, the CSV Agent still doesn't produce a CSV File. |
Hi, What do you do to have the Formatting Agent able to read de KeyValue Storage agent ? I am stuck in that step. The KeyValue Agent cannot be a source of the Formatting Agent, or at least it does not appear in the list of sources |
In the EventFormatting Agent you set the KeyValue Storage under "Controllers" (NOT "Source"). |
Thanks @radry I have my agents as you mention but without success: I have an event that gets data from a webpage and send it to a Key Value Storage: as shown in the image, data is arriving ok to the KVS agent and the EventFormating Agent is set as Controller Target. Int the EventFormating agent I have: But when dry-running the EventFormating Agent I receive an error message: When running in normal mode the first agent (webpage) nothing happens with the Formatting agent. Any idea what I am missing ? |
I don't remember exactly how I did and I don't have the scenario in its original state anymore. If I remember correctly dry run doesn't work in this case, that's normal. You need to use an external (manual) trigger or send an actual event. Something must trigger the Event Formatting agent, only then it will pull data from the KVS agent. Also I suggest you enable "propagate immediately" (it makes debugging easier) |
Thans for both, your attention and your suggestion. You are right, I defined a Manual Agent that has as a Receiver the EventFormatting agent and it worked. Data flows form KVS agent to Formatting agent. However, I arrived to the same problem you reported, data comes in a non useful format. If you know the name of the key, you can get the value, for example in my case I have in the KVS agent: variable: {{ var }} and one of my key-value pairs is: "MX:30" In the eventformatting agent I can extract only the value of that key with: {{ var.MX }} or {{ var["MX"] }} The result is only "30" without all the other non useful characters. But I do not know how to extract the key name itself. In my case keys are always the same, so this is not a problem, I can do: text: MX,{{ var.MX }}{ % line_break %} Note: the above goes everything in one single line, without line breaks. In case I would like to have it in CSV format. But for that purpose I guess that it is easier to have the data generating(retrieving) agents (i.e. Website Agent) writing directly to a file with the Local File Agent, adding one line for each event with a CSV structure, without using at all the KVS agent. After that, if needed, the CSV agent can receive that file and process line by line or all the lines in bulk. Thanks again !! |
I'm trying to read (serialize) data with a csv agent provided by key value storage agent.
The problem is that the data is always empty.
Key Value Store Agent control target is set to the CSV Agent.
In the CSV Agent "Controllers" is set to the Key Value Store Agent.
So Data exchange should be possible.
Key Value Storage Agent:
Sample Data:
CSV Agent:
I tried several different values for "data_path" and "use_field" but it's always empty (To be more precise the values it outputs are "\n\n" )
The text was updated successfully, but these errors were encountered: