I'm performing a small benchmark on an Elastic Search cluster. My benchmark script is written in bash and uses curl.
I'm writing the data to a file that I'm posting to the REST API:
curl -XPOST 'localhost:9200/benchmark/external/_bulk?pretty' \
--data #$DATAFILE
My $DATAFILE is very simple, and has all newlines in place:
{"index":{"_id": "1"}}
{"data":"xxxxxxxxxx"}
{"index":{"_id": "2"}}
{"data":"xxxxxxxxxx"}
{"index":{"_id": "3"}}
{"data":"xxxxxxxxxx"}
...
But when I try to do my post I keep receiving the following error:
{
"error" : "ActionRequestValidationException[Validation Failed: 1: no requests added;]",
"status" : 400
}
I understand that my input is not validated, but why?
curl removed the newlines before the data was sent!
The --data parameter should be replaced with --data-binary:
curl -XPOST 'localhost:9200/benchmark/external/_bulk?pretty' \
--data-binary #$DATAFILE
Related
I am hitting a wall trying to build a script to save myself quite a good bit of time. I am working in a system in which I need to run a curl POST against a list of values. The list is about 400 lines long, so I am hoping to find a way of scripting this in Bash instead of running that call manually for each entry. Below are some details to help understand what I'm trying to accomplish:
If I were to be doing this task manually, each call would be formatted like the below:
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
This points to my JSON in the listed file which shows as the below:
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": "apple"
}
If I run the above, the call works, and the expected operation occurs. The issue is that I need to run this against roughly 400 values for that "memberInfo" field in the JSON payload. I'm trying to identify a way to run a single bash script, which will run this curl command over and over, and update the JSON payload to use each row in a file as the below:
memberList.txt
apple
banana
peach
pear
orange
.
.
And then maybe insert a pointer in my JSON for the "memberInfo" field over to this file.
Any and all help/suggestions are greatly appreciated!
.
This will do as you intend. Its a little convoluted but you might polish it a bit.
#!/bin/bash
function getString(){
echo $1 | python3 -c '
import json
import sys
payload="""
{
"groupId": "12345678987654321",
"type": "serial",
"memberInfo": ""
}
"""
obj = json.loads(payload)
obj["memberInfo"] = sys.stdin.read().strip()
print(json.dumps(obj, indent = " "))
'
}
while read member
do
getString "$member" > json_payload.json
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d #json_payload.json
done <<< "$( cat fruits.txt )"
Hope it helps!
while read member; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '{"groupId": "12345678987654321","type": "serial","memberInfo": "$member"}'
done <members.txt
This will work if you only care about the memberInfo field, another method could be writing your json line by line to payloads.txt file.
payloads.txt
{"groupId": "12345678987455432","type": "stereo","memberInfo": "apple"}
{"groupId": "34532453453453465","type": "serial","memberInfo": "banana"}
...
then use this as the script
while read payload; do
curl -X POST --header "Content-Type: application/json" -v 'http://www.website.com:8081/cc/membership' -d '$payload'
done <payloads.txt
here is a collection of bash scripting common uses I've had to use
https://github.com/felts94/advanced-bash/blob/master/bash_learn.sh
I want to send an email with attached pdf file through the Sparkpost API with curl post.
To insert the pdf I use (my test.pdf is ~ 200KB)
"data":"'$(cat test.pdf} | base64 --wrap=0)'"
But somehow this doesn't work out showing the following error:
/usr/bin/curl: Die Argumentliste ist zu lang (original)
/usr/bin/curl: Argument list is too long
EDIT:
curl command
curl -X POST https://api.eu.sparkpost.com/api/v1/transmissions -H 'Authorization: <APIKEY>' -H 'Content-Type: application/json' -d '{
"options":{
"open_tracking":false,
"click_tracking":false,
"inline_css":false
},
"recipients":[
{
"address":{
"email":"user#domain.tld",
"name":"user"
}
}
],
"content":{
"from":{
"name":"sender",
"email":"sender#domain.tld"
},
"reply_to":"replyto#domain.tld",
"subject":"subject",
"text":"textbody",
"attachments":[
{
"name":"attachmentname.pdf",
"type":"application/pdf",
"data":"'$(cat test.pdf | base64 --wrap=0)'"
}
]
}
}'
This is coming up because you are trying to pass the entirety of the base64'd content on the command line. curl has the ability to load in data to POST from a file, which I'd recommend doing. More information can be found in the man page, but the basic format is this:
curl -X POST -d #filename.txt https://website.com/path
According to the curl manual, the -F option allows you to encode a file for base64, but limits the output to 76 characters.
Ex:
-F '=#localfile;encoder=base64'
Hey there i am using the most recent command line tools from monero:
monerod:
Monero 'Lithium Luna' (v0.12.3.0-release)
Started using: ./monerod
Running on: 127.0.0.1:18081
monero-wallet-rpc:
Monero 'Lithium Luna' (v0.12.3.0-release)
Started using: ./monero-wallet-rpc --wallet-dir /path/to/wallets/ --rpc-bind-port 28081 --disable-rpc-login
Running on: 127.0.0.1:28081
I am currently testing rpc calls from:
https://getmonero.org/resources/developer-guides/wallet-rpc.html
Most of them work, but there are many rpc calls which seem not working...
For example:
curl -X POST http://localhost:28081/json_rpc -d '{"jsonrpc":"2.0","id":"0","method":"refresh"}' -H 'Content-Type: application/json'
Returns:
{
"error": {
"code": -32601,
"message": "Method not found"
},
"id": "0",
"jsonrpc": "2.0"
}
Can anyone refer to this?
I am using the most recent versions and the documentation is also up to date.
Thanks and Greetings!
Try this command with the params option in the json.
curl -X POST http://localhost:28081/json_rpc \
-d '{"jsonrpc":"2.0","id":"0","method":"refresh","params":{"start_height":100000}}' \
-H 'Content-Type: application/json'
The problem is that you're using an --rpc-bind-port equivalent to that of the testnet wallet RPC port. Try using another value like 40000 and curl to that instead.
The relevant ports are here under the namespace 'config':
https://github.com/monero-project/monero/blob/master/src/cryptonote_config.h
Am working on Elasticsearch server and using the curl command for posting, getting the data in Windows command line.
When I try to post the data using the curl -XPUT command the data appears getting inserted. But when I query the data back using curl-XGET I am not getting the data but only header information like Index number etc. Please see the queries and results below.
curl -XPUT "<server location>/megacorp/emp/1" -d "{""first_name"" : ""John"",""last_name"" : ""Smith"",""age"" : "25"}"
{"_index":"megacorp","_type":"emp","_id":"1","_version":1,"created":true}
curl -XPUT "<server location>/megacorp/emp/2" -d "{""first_name"" : ""Jane"",""last_name"" : ""Cooper"",""age"" : "35"}"
{"_index":"megacorp","_type":"emp","_id":"2","_version":1,"created":true}
curl -XPUT "<server location>/megacorp/emp/3" -d "{""first_name"" : ""Bradleey"",""last_name"" : ""Cooper"",""age"" : "40"}"
{"_index":"megacorp","_type":"emp","_id":"3","_version":1,"created":true}
curl -XGET "<server location>/megacorp/emp/_search?q=last_name:Cooper"
{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":2,"max_score":1.0,"hits":[{"_index":"megacorp","_type":"emp","_id":"2","_score":1.0},{"_index":"me
gacorp","_type":"emp","_id":"3","_score":1.0}]}}
Try this: curl -XGET "<server location>/megacorp/emp/_search?q=last_name:Cooper&_source=true"
I"m using the following commands which i've copied over from the documentation. Assume that I'm passing in the application id and keys in each curl statement for brevity.
create player
curl -X POST \
-d '{"name":"abhinav", "rank" :"amazing"}' \
https://api.parse.com/1/classes/Player
Player objectId: HgMZF6H90L
Create game and add pointer relation
curl -X POST \
-d '{"Level" : "TWO"}' \
https://api.parse.com/1/classes/GameScore
GameScore objectId: cwYIEwFaq9
curl -X PUT \
-d '{"opponents":{"__op":"AddRelation","objects":[{"__type":"Pointer","className":"Player","objectId":"HgMZF6H90L"}]}}' \
https://api.parse.com/1/classes/GameScore/tDiFZSE0lQ
Now I check from the Data browser and the GameScore object has a field with Relation. clicking on the "View Relations" navigates me to the correct list underneath. Working fine except that it says "Relation" and not pointer. Not sure if that is relevant
HOWEVER, I'm not able to query this information from the REST API. A GET results in the relation data but no objectId for the player available.
curl -X GET \
https://api.parse.com/1/classes/GameScore/cwYIEwFaq9
result:
{"Level":"TWO","createdAt":"2014-06-11T08:49:43.325Z","objectId":"cwYIEwFaq9","opponents":{"__type":"Relation","className":"Player"},"updatedAt":"2014-06-11T08:51:01.093Z"}
tried with include option but that results in some error
curl -X GET \
--data-urlencode "include=opponents" \
https://api.parse.com/1/classes/GameScore/cwYIEwFaq9
result:
code 107, invalid JSON
What am I doing wrong?
why are you using "Remove" to add a relation? Look at the docs again.
go to Docs section "Creating Roles" again....
"__op": "AddRelation",
"objects": [
{
"__type": "Pointer",
"className": "_Role",
"objectId": "Ed1nuqPvc"
}
]
-- EDIT
to get using "include=opponents" you could create with this:
{"opponents":{"__op":"Add","objects":[{"__type":"Pointer","className":"_Role","objectId":"Ed1..."}]}}
Solved :
The problem is that i needed to create an array of pointers. So instead of the "AddRelation", i simply need to use the "AddUnique" operation to create an entry for the array. So
curl -X POST \
-d '{"Level" : "TWO", "opponents":{"__op":"AddUnique","objects":[{"__type":"Pointer","className":"Player","objectId":"5Q4QsKF8QR"}]}}' \
https://api.parse.com/1/classes/GameScore