Missing newline for adding with bulk API - elasticsearch

Question

I want to add the following file to Elasticsearch using the bulk API:
{"_id":{"date":"01-2007","profile":"Da","dgo":"DGO_E_AIEG","consumerType":"residential"},"value":{"min":120.42509,"minKwh":0.20071,"nbItems":6.0}}
using the command
curl -XPOST -H 'Content-Type: application/json' localhost:9200/_bulk --data-binary Downloads/bob/test.json
but I got the following mistake:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\n]"}],"type":"illegal_argument_exception","reason":"The bulk request must be terminated by a newline [\n]"},"status":400}
NB: The file clearly has a empty line at the end

Answer 1

In the docs it says:
NOTE: the final line of data must end with a newline character \n.
There is an example above that of what the document is expected to look like. https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html. Perhaps adding \n at the end of each line would fix the issue.
UPDATE:
There might be something wrong with the way you have placed your data into your JSON file. For example, the following data is in example.json:
{ "index" : { "_index" : "example", "_type" : "doc", "_id" : "1" } }
{ "field1" : "value1" }
<space here>
When running the following curl command, it works:
curl -X POST -H "Content-Type: application/x-ndjson" localhost:9200/_bulk --data-binary "#example.json"
It could be that you're not including something important in your JSON file, or you don't have "#your_file.json", or like the other poster mentioned, you don't have the content-type as application/x-ndjson.

Answer 2

The answer is very simple
{ "index":{ "_index":"schools_gov", "_type":"school", "_id":"1" } }
{ "name":"Model School", "city":"Hyderabad"}
{ "index":{ "_index":"schools_gov", "_type":"school", "_id":"2" } }
{ "name":"Government School", "city":"Pune"}
is not going to work but the below json will work
{ "index":{ "_index":"schools_gov", "_type":"school", "_id":"1" } }
{ "name":"Model School", "city":"Hyderabad"}
{ "index":{ "_index":"schools_gov", "_type":"school", "_id":"2" } }
{ "name":"Government School", "city":"Pune"}
//Give a new line here. Not '\n' but the actual new line.
The HTTP command would be POST http://localhost:9200/schools_gov/_bulk

Answer 3

As the error states, you simply need to add a new line to the end of the file.
If you are on a *nix system, you can do this:
echo "\n" >> Downloads/bob/test.json
Also, as explained in the documentation https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html, the Content-Type should be application/x-ndjson
When sending requests to this endpoint the Content-Type header should
be set to application/x-ndjson
So the command should be:
curl -XPOST -H 'Content-Type: application/x-ndjson' localhost:9200/_bulk --data-binary Downloads/bob/test.json

Answer 4

The error message is very confusing. I typed -data-binary and got the same message. The message sent me to completely wrong direction.