Elastic Search Bulk API - Not indexing - elasticsearch

I have output a JSON file in bulk format which I can load in to Kibana with the developer tools. and by inserting a few lines using the -d command
example lines of file:
{"index":{"_index":"els","_type":"logs","_id":1481018400003}}
{"timestamp":1481018400003,"zoneId":29863567,............[]}
{"index":{"_index":"els","_type":"logs","_id":"30cee368073c0c9b"}}
{"timestamp":1481018400005,"zoneId":29863567,............[]}
...
However when I run the bulk api to pot a file it does not do anything. I added verbose to the command and get the following:
* Connected to localhost (::1) port 9200 (#0)
> POST /_bulk HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.49.0
> Accept: */*
> Content-Length: 0
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 400 Bad Request
< content-type: application/json; charset=UTF-8
< content-length: 165
* HTTP error before end of send, stop sending
Any help would be great.
Thanks!

Related

can't use curl to query neo4j

I am trying to use curl to query neo4j
curl -X POST -H Accept:application/json -H Content-Type:application/json -u neo4j:password -v http://localhost:7474/db/neo4j/tx/commit -d '{"statements":[{"statement":"MATCH (n) RETURN n"}]}'
gives me this response
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 127.0.0.1:7474...
* Connected to localhost (127.0.0.1) port 7474 (#0)
* Server auth using Basic with user 'neo4j'
> POST /db/neo4j/tx/commit HTTP/1.1
> Host: localhost:7474
> Authorization: Basic bmVvNGo6cGFzc3dvcmQ=
> User-Agent: curl/7.79.1
> Accept:application/json
> Content-Type:application/json
> Content-Length: 47
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 27 Jul 2022 09:13:35 GMT
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Content-Length: 120
<
{"results":[],"errors":[{"code":"Neo.ClientError.Request.InvalidFormat","message":"Could not parse the incoming JSON"}]}* Connection #0 to host localhost left intact
If anyone could help please
Should have mentioned I'm on windows. Apparently you have to escape those double quotes in the json
This works for me now:
curl -X POST -H Accept:application/json -H Content-Type:application/json -u neo4j:password -v http://localhost:7474/db/neo4j/tx/commit -d "{\"statements\":[{\"statement\":\"MATCH (n) RETURN n\"}]}"

How to access Gravitee AM api

How to curl the basic gravitee am api? i tried the simple one also cant?
curl -X POST http://localhost:8093/admin/token -H 'authorization: Basic base64(admin:adminadmin)'
curl -X POST http://localhost:8093/admin/token -H 'Authorization: Basic YWRtaW46YWRtaW5hZG1pbg=='
It stated 404 Not found. Result are below:
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8093 (#0)
> POST /admin/token HTTP/1.1
> Host: localhost:8093
> User-Agent: curl/7.58.0
> Accept: */*
> Authorization: Basic YWRtaW46YWRtaW5hZG1pbg==
>
< HTTP/1.1 404 Not Found
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Type: text/html;charset=iso-8859-1
< Content-Length: 0
<
* Connection #0 to host localhost left intact
First you have to request an access token:
curl -X POST http://localhost:8093/management/auth/token -u admin:adminadmin
You should receive this payload:
{"access_token":"eyJraWQiOiJkZWZ......TyJ37E","expires_at":"Mon Nov 29 15:06:12 CET 2021","token_type":"bearer"}
Then you can use this access_token to request the access management API.
curl -X GET http://localhost:8093/management/organizations/DEFAULT/environments/DEFAULT/domains/ -H'Authorization: Bearer eyJraWQiOiJkZWZ......TyJ37E'

Elastic search query to return the version of kibana

I am running on kibana 5.4.0
I wanted to know if there is an Elastic search GET api query that I can run against the .kibana index which returns the version of kibana I am running on?
Why query elasticsearch? Kibana itself offers that capabiltity:
cinhtau#omega:~> curl -v localhost:5601/status
* About to connect() to localhost port 5601 (#0)
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5601 (#0)
> GET /status HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost:5601
> Accept: */*
>
< HTTP/1.1 302 Found
< location: https://localhost:5601/status
< kbn-name: kibana
< kbn-version: 5.6.1
< kbn-xpack-sig: 83f97b6a01fc027688f430e60e935b27
< cache-control: no-cache
< content-length: 0
< Date: Fri, 22 Sep 2017 08:51:43 GMT
< Connection: keep-alive
<
* Connection #0 to host localhost left intact
From the response header you can see it is 5.6.1. There is a way to query it from Elasticsearch, but IMHO doesn't make sense to ask Elasticsearch, information about Kibana.

Using PDF Reactor as Web Service

I am discovering PDF reactor and I'd like to use it as a web service. To test a file, I use cURL
curl -v -X POST --header "Content-Type:application/xml" http://localhost:9423/service/rest/convert/async -d #test.html
Is that correct ?
test.html :
<html>
<body>
Coucou, je suis terrien.
</body>
</html>
Thank you for your help,
Cédrik
edit #1:
response from the comman above :
* About to connect() to localhost port 9423 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9423 (#0)
> POST /service/rest/convert/async HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:9423
> Accept: */*
> Content-Type:application/xml
> Content-Length: 50
>
< HTTP/1.1 400 Bad Request
< Content-Type: text/plain
< Date: Tue, 15 Dec 2015 11:47:29 GMT
< Content-Length: 307
< Server: Jetty(9.3.2.v20150730)
<
* Connection #0 to host localhost left intact
* Closing connection #0
JAXBException occurred : élément inattendu (URI : "", local : "html"). Les éléments attendus sont <{http://webservice.pdfreactor.realobjects.com/}configuration>. élément inattendu (URI : "", local : "html"). Les éléments attendus sont <{http://webservice.pdfreactor.realobjects.com/}configuration>.
When using the REST API of PDFreactor via cURL you have to send a configuration XML or JSON to the server which includes configuration for PDFreactor and your document, as described here: http://www.pdfreactor.com/product/doc_html/index.html#d0e688
A sample configuration for XML could look like this:
config.xml:
<tns:configuration xmlns:tns="http://webservice.pdfreactor.realobjects.com/">
<document><html> <body> Coucou, je suis terrien. </body> </html></document>
</tns:configuration>
You can then call the following:
curl -v -X POST --header "Content-Type:application/xml" http://localhost:9423/service/rest/convert/async.xml -d #config.xml
The output will look like the following:
* About to connect() to localhost port 9423
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9423
> POST /service/rest/convert/async.xml HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: localhost:9423
> Accept: */*
> Content-Type:application/xml
> Content-Length: 195
>
> <tns:configuration xmlns:tns="http://webservice.pdfreactor.realobjects.com/"> <document><html><body>Coucou, je suis terrien.</body></html></document></tns:configuration>HTTP/1.1 202 Accepted
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Headers: Accept, Content-Length, content-type, Host, User-Agent
< Access-Control-Allow-Methods: GET, PUT, POST, DELETE
< Access-Control-Expose-Headers: Location
< Cache-Control: no-cache
< Date: Wed, 16 Dec 2015 16:34:19 GMT
< Location: http://localhost:9423/service/rest/progress/c2a58dbd-ef9d-4b79-87d9-079c139fe9ed
< Content-Length: 0
< Server: Jetty(9.3.2.v20150730)
* Connection #0 to host localhost left intact
* Closing connection #0
The "Location" response header contains the URL which can be used to retrieve the progress of the conversion, so you can retrieve the progress with (the ID will of course vary):
curl -v http://localhost:9423/service/rest/progress/c2a58dbd-ef9d-4b79-87d9-079c139fe9ed
This will return the conversion progress and if the conversion has finished the "Location" repsonse header will contain a new URL to retrieve the document. You can use ".pdf" to retrieve the PDF binary data or ".xml" to retrieve XML data containing the PDF as base64 encoded String, the number of pages of the document, etc.
curl -v http://localhost:9423/service/rest/document/c2a58dbd-ef9d-4b79-87d9-079c139fe9ed.pdf

Changing HTTP status message using Sinatra

I'm writing a simple Sinatra app, and given a user posts a request with an specific data, I want to return an error '453' (custom error code) with a message CLIENT_ERROR, or something similar.
The problem is: looking into the Sinatra documentation and doing some testing I couldn't find a way to setup the response error message, only the response status.
So, if a set the Sinatra response
get '/' do
response.status = 453
end
I get the error code right:
curl -v localhost:4567
* About to connect() to localhost port 4567 (#0)
* Trying 127.0.0.1... connected
> GET / HTTP/1.1
> User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: localhost:4567
> Accept: */*
>
< HTTP/1.1 453
< X-Frame-Options: sameorigin
< X-XSS-Protection: 1; mode=block
< Content-Type: text/html;charset=utf-8
< Content-Length: 0
< Connection: keep-alive
< Server: thin 1.3.1 codename Triple Espresso
<
* Connection #0 to host localhost left intact
* Closing connection #0
But what I want to have is:
< HTTP/1.1 453 CLIENT_ERROR
The same way I have
< HTTP/1.1 200 OK
When everything goes according to the plan.
Is there anyway to do this using Sinatra/Rack?
The status message is generated by the server you are using, e.g. in Thin the messages are in Thin::HTTP_STATUS_CODES and the reponse line is generated in Thin::Response, and in WEBrick they are in WEBrick::HHTPStatus::StatusMessage and the response is generated in WEBrick::HTTPResponse.
If you know what server you are using, you could add your error to the appropriate hash.
With Thin:
require 'thin'
Thin::HTTP_STATUS_CODES[453] = "Client Error"
and the output:
$ curl -v localhost:4567
* About to connect() to localhost port 4567 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 4567 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
> Host: localhost:4567
> Accept: */*
>
< HTTP/1.1 453 Client Error
< X-Frame-Options: sameorigin
< X-XSS-Protection: 1; mode=block
< Content-Type: text/html;charset=utf-8
< Content-Length: 0
< Connection: keep-alive
< Server: thin 1.4.1 codename Chromeo
<
* Connection #0 to host localhost left intact
* Closing connection #0
and with WEBrick:
require 'webrick'
WEBrick::HTTPStatus::StatusMessage[453] = "Client Error"
which gives the output:
$ curl -v localhost:4567
* About to connect() to localhost port 4567 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 4567 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8r zlib/1.2.3
> Host: localhost:4567
> Accept: */*
>
localhost - - [13/Aug/2012:01:41:48 BST] "GET / HTTP/1.1" 453 0
- -> /
< HTTP/1.1 453 Client Error
< X-Frame-Options: sameorigin
< X-Xss-Protection: 1; mode=block
< Content-Type: text/html;charset=utf-8
< Content-Length: 0
< Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-04-20)
< Date: Mon, 13 Aug 2012 00:41:48 GMT
< Connection: Keep-Alive
<
* Connection #0 to host localhost left intact
* Closing connection #0
I would recommend not to use custom HTTP status codes. If you think you have something of general use, consider writing an Internet Draft and going through the IETF specification process.

Resources