Using the v2 of the box api, I use the folder items request to get information on files in a folder: http://developers.box.com/docs/#folders-retrieve-a-folders-items
I'm looking at trying to parse the response data. Any ideas how I can do this in bash to easily find a file in the user's account? I would like to find the name of the file where I can get the ID of the file as well.
response looks something like this:
{
"total_count": 25,
"entries": [
{
"type": "file",
"id": "531117507",
"sequence_id": "0",
"etag": "53a93ebcbbe5686415835a1e4f4fff5efea039dc",
"name": "agile-web-development-with-rails_b10_0.pdf"
},
{
"type": "file",
"id": "1625774972",
"sequence_id": "0",
"etag": "32dd8433249b1a59019c465f61aa017f35ec9654",
"name": "Continuous Delivery.pdf"
},
{ ...
For bash, you can use sed or awk. Look at Parsing JSON with Unix tools.
Also if you can use a programming language, then python can be your fastest option. it has a nice module json http://docs.python.org/library/json.html. It has a simple decode API which will give a dict as the output
Then
import json
response_dict = json.loads(your_response)
I recommend using jq for parsing/munging json in bash. It is WAY better than trying to use sed or awk to parse it.
Related
I want my output to be as below:-
{
"EVENT": "start",
"NAME": "schedule-rule-start",
"ECS_CLUSTER": "gps-aws-infra-ecs-cluster",
"ECS_SERVICE_NAME": [
“abc”,
"def"
],
"DESIRED_COUNT": "1",
"REGION": “eu-central-1"
}
My Input in yaml is as below:
Input: !Sub '{"EVENT": "${self:custom.input.StartECSServiceRule.Event}", "NAME": "${self:custom.input.StartECSServiceRule.Name}", "ECS_CLUSTER": "${self:custom.input.StartECSServiceRule.ClusterName}", "ECS_SERVICE_NAME": "${self:custom.input.StartECSServiceRule.ECSServiceName}","DESIRED_COUNT": "${self:custom.input.StartECSServiceRule.DesiredCount}","REGION": "${self:provider.region}"}'
I want the generated cloudformation json as below:-
“{\“EVENT\“:\“start\“,\“NAME\“:\“schedule-rule-start\“,\“ECS_CLUSTER\“:\“gps-aws-infra-ecs-cluster\“,\“ECS_SERVICE_NAME\“:["abc","def"],\“DESIRED_COUNT\“:\“1\“,\“REGION\“:\“eu-central-1\“}”
Need help on how should i provide the value of "ECSServiceName" as below doesnot work in the Input I gave in the yaml file???
"${self:custom.input.StartECSServiceRule.ECSServiceName}
Please help to correct my yaml file Input. Right now it comes as string , i want it to come as array.
I my Nifi controller I want to configure the FreeFormTextRecordSetWriter, but I have no Idea what I should put in the "Text" field. I'm getting the text from my source (in my case GetSolr), and just want to write this, period.
Documentation and mailinglist do not seem to tell me how this is done, any help appreciated.
EDIT: Here the sample input + output I want to achieve (as you can see: not ransformation needed, plain text, no JSON input)
EDIT: I now realize, that I can't tell GetSolr to return just CSV data - but I have to use Json
So referencing with attribute seems to be fine. What the documentation omits is, that the ${flowFile} attribute should containt the complete flowfile that is returned.
Sample input:
{
"responseHeader": {
"zkConnected": true,
"status": 0,
"QTime": 0,
"params": {
"q": "*:*",
"_": "1553686715465"
}
},
"response": {
"numFound": 3194,
"start": 0,
"docs": [
{
"id": "{402EBE69-0000-CD1D-8FFF-D07756271B4E}",
"MimeType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"FileName": "Test.docx",
"DateLastModified": "2019-03-27T08:05:00.103Z",
"_version_": 1629145864291221504,
"LAST_UPDATE": "2019-03-27T08:16:08.451Z"
}
]
}
}
Wanted output
{402EBE69-0000-CD1D-8FFF-D07756271B4E}
BTW: The documentation says this:
The text to use when writing the results. This property will evaluate the Expression Language using any of the fields available in a Record.
Supports Expression Language: true (will be evaluated using flow file attributes and variable registry)
I want to use my source's text, so I'm confused
You need to use expression language as if the record's fields are the FlowFile's attributes.
Example:
Input:
{
"t1": "test",
"t2": "ttt",
"hello": true,
"testN": 1
}
Text property in FreeFormTextRecordSetWriter:
${t1} k!${t2} ${hello}:boolean
${testN}Num
Output(using ConvertRecord):
test k!ttt true:boolean
1Num
EDIT:
Seems like what you needed was reading from Solr and write a single column csv. You need to use CSVRecordSetWriter. As for the same,
I should tell you to consider to upgrade to 1.9.1. Starting from 1.9.0, the schema can be inferred for you.
otherwise, you can set Schema Access Strategy as Use 'Schema Text' Property
then, use the following schema in Schema Text
{
"name": "MyClass",
"type": "record",
"namespace": "com.acme.avro",
"fields": [
{
"name": "id",
"type": "int"
}
]
}
this should work
I'll edit it into my answer. If it works for you, please choose my answer :)
I wish to use the JoltTransformJSON spec that can be used to convert the input to output.
I have tried to use map to List and other syntax, but was not been successful so far.
Expected input:
{
"params": "sn=GH6747246T4JLR6AZ&c=QUERY_RECORD&p=test_station_name&p=station_id&p=result&p=mac_addresss"
}
Expected output:
{
"queryType": "scan",
"dataSource": "xyz",
"resultFormat": "list",
"columns": ["test_station_name", "station_id", "result", "mac_address"],
"intervals": ["2018-01-01/2018-02-09"],
"filter": {
"type": "selector",
"dimension": "sn",
"value": "GH6747246T4JLR6AZ"
}
}
Except for the content inside Columns and dimension and value attributes rest of the fields are hardcoded.
As all of the data is contained in a single JSON key/value, I don't think JoltTransformJSON is the best option here. I actually think writing a simple script in Python/Groovy/Ruby to split the querystring value and write it out as JSON is easier and less complicated to maintain. I would recommend Groovy specifically (you can use the specialized ExecuteGroovyScript processor), as it is the most performant & robust in Apache NiFi and has excellent JSON handling.
i am trying to use campaigns/update in MailChimp.
However i still cannot understand how can i use it.
Can anyone please explain to me, for example how can i change the "from_email" parameter?
my current json for that purpose looks like this:
{
"apikey": "my_api_key",
"cid": "my_campaign_id",
"name": "options",
"value": ["list_id_value","subject_value","from_email_value"]
}
However when i send this json to campaigns/update, it returns error saying that "Can not update unknown option "2"".
How can i change the json so that i can update the "from_email"?
The API docs seem to be wrong. Try replacing [ ] with { } and giving field:pair values inside it. So, example to change the subject:
{
"apikey": "my_api_key",
"cid": "my_campaign_id",
"name": "options",
"value": {"subject": "My new subject"}
}
I have a JSON file:
{
"user": "John",
"first": "John",
"last": "Wilson",
"updated": "2013-02-17",
"generated_at": "2013-02-13",
"version": 1.1,
}
I want to use this as the data file for my Ruby test and want to access the data in this file. I am doing the data verification as:
application[data_first]. should eq 'John'
I want to refer to the expected data from the JSON file using something like:
application[data_first]. should eq JSON_file[first]
Assuming the JSON file's name is "JSON_file", I also have added require 'JSON_file' at the top of my test script.
How do I access the data from JSON file?