I have NiFi running on a test environment and would like it to automatically start all processors as the last step of deployment.
Do I need to parse all the processor ids and hit the /nifi-api/processors endpoint for each one or is it possible to start all - eg from the /nifi-api/process-groups endpoint ?
My NiFi flow has uuid 66f83c1d-0162-1000-baff-01e60296540a - this GET curl statement returns information successfully:
curl -i -X GET http://localhost:9090/nifi-api/process-groups/66f83c1d-0162-1000-baff-01e60296540a
However trying to PUT a status fails:
curl -i -X PUT -H 'Content-Type: application/json' -d '{"status":"RUNNING"}' http://localhost:9090/nifi-api/process-groups/66f83c1d-0162-1000-baff-01e60296540a
HTTP/1.1 400 Bad Request
Date: Mon, 06 Aug 2018 11:08:15 GMT
X-Frame-Options: SAMEORIGIN
Content-Type: text/plain
Vary: Accept-Encoding
Content-Length: 429
Server: Jetty(9.4.3.v20170317)
Cannot construct instance of `org.apache.nifi.web.api.dto.status.ProcessGroupStatusDTO` (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('RUNNING')
at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 11] (through reference chain: org.apache.nifi.web.api.entity.ProcessGroupEntity["status"])
Yes by using NiFi RestApi we can start all processors in a processor group.
As you are missing id key value in your json,
{"id":"66f83c1d-0162-1000-baff-01e60296540a","state":"RUNNING"}
we need to mention state not status to run/stop processors in the group
{"id":"66f83c1d-0162-1000-baff-01e60296540a","state":"RUNNING"}
and we need to include flow in the address of the rest api.
http://localhost:9090/nifi-api /flow/ process-groups/66f83c1d-0162-1000-baff-01e60296540a
Try with below Curl command
curl -i -X PUT -H 'Content-Type: application/json' -d '{"id":"66f83c1d-0162-1000-baff-01e60296540a","state":"RUNNING"}' http://localhost:9090/nifi-api/flow/process-groups/66f83c1d-0162-1000-baff-01e60296540a
Refer to this for more details regarding start process group using RestAPI.
Related
From what I've read on https://graphql.org/learn/serving-over-http/ and Performance test for graphQL API, I can send a graphQL request over jmeter.
In the stack overflow example, a graphQL server was built with Apollo and then queried via Jmeter. I am just trying to do load testing and not build schemas or data sets, so I don't have Apollo on my local machine, and I'm also not sure that I would be testing the QA environment if I did try to do this all locally.
I can access our graphQL at https://proprietary-website/graphql/gql and if I run the following query, it returns the expected results:
query currentUserProfile {
currentUserProfile {
id
}
}
Now, I try to apply that to jmeter
I create a HTTP Request in my thread group with protocol: https, server name: proprietary-website, method: GET, path: /graphql/gql
I create an HTTP Header Manager and include the required Bearer Token and Content-Type headers
I create a listener to View Results Tree
Then I run a jmeter load test with 500 threads
I see the valid URL passed in the request body:
GET https://proprietary-website/graphql/gql
GET data:
{"query":{"currentUserProfile {id}"}}
[no cookies]
but I get back:
HTTP/1.0 400 Bad Request
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 1555
Date: Thu, 12 Dec 2019 16:40:26 GMT
I am new to graphql, am I not allowed to send Jmeter requests to the URL my dev gave me and do I have to use the apollo server? If so, can someone point me to some examples?
I am on Windows 10
I also tried:
k6, but this requires docker and i have no admin rights on my computer
easygraphql-lt but when I ran a test script with #easygraphql-lt, I got dependency errors that I don't quite understand how to fix
I looked at easygraphql-load-tester, but this would require me to have access to the project but all i have is the website. I don't have a graphQL.js or an index.js to reference
UPDATE: I just saw that I can copy the query as a curl request so I pasted that request in command line and it gave me extra headers that I put in my jmeter HTTP Header manager:
$ curl 'https://proprietary-website/graphql/gql' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://proprietary-website' -H 'Authorization: Bearer redacted' --data-binary '{"query":" query currentUserProfile { currentUserProfile { id }}"}'
But I still get a 400 back, so I think I need to send the query differently as it is showing appended as a --data-binary (not sure what that means). I tried some other SO suggestions, no luck so far
I also tried against an app out of the O'Reilly graphql book, snowtooth.herokuapp.com. This was simpler, so the curl request was simpler, and the information I pasted into jmeter was less, but I still got the error: GET query missing using Protocol: HTTP, server name: snowtooth.herokuapp.com, method: GET and Body Data: {"query":"query { allLifts { name}}"}, plus all the requisite headers in the Header Manager
Not sure what your proprietary website implementation is, however this http://snowtooth.herokuapp.com/ can be tested like:
HTTP GET request:
HTTP POST request:
Also be aware that since JMeter 5.1 it's possible to create a JMeter test plan from CURL command, this http://snowtooth.herokuapp.com/ gives me the following CURL command:
curl 'http://snowtooth.herokuapp.com/' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: http://snowtooth.herokuapp.com' --data-binary '{"query":"{allLifts{name}}"}' --compressed
And when I import it to JMeter I can execute the requests successfully:
From "Test in Web Chat" window in Azure Dashboard, I entered "hello" which is correctly received in my backend, but I'm still figuring out how to reply.
Note: WORDS IN CAPS means variable.
curl -d "grant_type=client_credentials&client_id=CLI-ENT-ID&client_secret=SECRET&scope=https%3A%2F%2Fapi.botframework.com%2F.default" -X POST https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token
Response: {"token_type":"Bearer","expires_in":3600,"ext_expires_in":0,"access_token":"VERY_LONG_STRING"}
curl -d '{"type":"message","from":{"id":"mybot#SOME_ID","name":"mybot"},"conversation":{"id":"CONVERSATION_ID"},"recipient":{"id":"RECIPIENT_ID","name":"You"},"locale":"en","text":"hello too","replyToId":"CONVERSATION_ID|0000008"}' -H "Content-Type: application/json" -H "Authorization: Bearer VERY_LONG_STRING" -X POST https://api.botframework.com/v3/conversations/CONVERSATION_ID/activities/CONVERSATION_ID%7C0000008
Response: The page cannot be displayed because an internal server error has occurred.
Request from my app (also same error):
POST /v3/conversations/CONVERSATION_ID/activities/CONVERSATION_ID%7C0000008 HTTP/1.1
Accept: application/json
Content-Type: application/json
Authorization: Bearer VERY_LONG_STRING
User-Agent: Java-SDK
Content-Length: 273
Host: api.botframework.com
Connection: Keep-Alive
Accept-Encoding: gzip,deflate
{"type":"message","from":{"id":"mybot#SOME_ID","name":"mybot"},"conversation":{"id":"CONVERSATION_ID"},"recipient":{"id":"RECIPIENT_ID","name":"You"},"locale":"en","text":"hello too","replyToId":"CONVERSATION_ID|0000008"}
What do I miss?
Thanks to comment from Eric Dahlvang, I figure it out.
By default, ApiClient generated from swagger will have its basePath hardcoded to https://api.botframework.com.
Before replying, take the serviceUrl from incoming Activity and use it to change the basePath.
I am building a script that uses a cURL command against an API. I send the cURL command formatted as an application/json request and get the result, which I parse into a Ruby hash.
This works great when I use cURL POST commands, getting the correctly formatted JSON responses. However, when using cURL GET commands I am returned a JSON document that has headers:
puts r:
HTTP/1.1 200 OK
X-Compute-Request-Id: req-7e625990-068b-47d1-8c42-9d3dd3b27050
Content-Type: application/json
Content-Length: 1209
Date: Wed, 16 Jan 2013 20:47:41 GMT
{ <JSON DATA> }
When I try and do a JSON.parse(r) I get an unexpected token error at 'HTTP/1.1.'.
My method for this:
def list_flavors
r = %x(curl -s -k -D - -H \"X-Auth-Token: #{$token}\" -X 'GET' http://10.30.1.49:8774/v2/27e60c130c7748f48b0e3e9175702c30/flavors -H 'Content-type: application/json')
response = JSON.parse(r)
response
end
Is there a way to use regular expressions to pull the body out of the JSON doc and then parse?
Or am I going about this the wrong way when getting the response from cURL?
You'll need to find a way to cut out that header before passing the string into JSON.parse. JSON.parse expects valid json only.
Rather than curling and using the thing wholesale as a string, I'd suggest you use the very versatile ruby Net::HTTP and/or OpenURI libraries, which will allow you to easily access just your response's body without the header.
Just been tinkering with Sinatra and trying to get a bit of a restful web service going.
The error I'm getting at the moment is very specific though.
Take this example post method
post '/postMan/:someParam' do
#Edited here. This code can be anything. 411 is still the response
puts params[:someParam]
end
Seems simple enough. Take a param, make an object out of it, then go store it in whatever way the objects save method defines.
Heres what I use to post the data using Curl
$curl -I -X POST http://127.0.0.1/postman/123456
The only problem is, I'm getting 411 back and have no idea why.
To the best of my knowledge, 411 is length required. Here is the trace
HTTP/1.1 411 Length Required
Content-Type: text/html; charset=ISO-8859-1
Server: WEBrick/1.3.1 (Ruby/1.9.2/2011-07-09)
Date: Fri, 02 Mar 2012 22:27:09 GMT
Content-Length: 303
Connection: close
I 'Cannot' change the curl message in any way. So might anyone have a way to set the content length to be ignored in sinatra? Or some fix which doesn't involve changing the curl request?
For the record, it doesn't matter whether I use the parameters in the Post method or not. I could have some crazy code inside it, it will still throw the same error
As others said above, WEBrick wrongly requires POST requests to have a Content-Length header. I just pass an empty body, because it's less typing than passing in the header:
curl -X POST -d '' http://webrickwhyyounotakeemptyposts.com/
Are you sure you're on port 80 for your app?
When I run:
ruby -r sinatra -e "post('/postMan/:someParam'){puts params[:someParam]}"
and curl it:
curl -I -X POST http://127.0.0.1:4567/postMan/123456
HTTP/1.1 200 OK
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
Content-Type: text/html;charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: thin 1.3.1 codename Triple Espresso
it's ok. Had to change the URL to postManthough, your example threw a 404because you had postman.
The output was also as expected:
== Sinatra/1.3.2 has taken the stage on 4567 for development with backup from Thin
>> Thin web server (v1.3.1 codename Triple Espresso)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:4567, CTRL+C to stop
123456
Ah. Try it without -I. It's probably sending a HEAD request and as such, not sending what you expect. Use -v if you want to show the headers.
curl -v -X POST http://127.0.0.1/postman/123456
curl -v -X POST -d "key=val" http://127.0.0.1/postman/123456
WEBrick erroneously requires POST requests to include the Content-Length header.
curl -H 'Content-Length: 0' -X POST http://example.com
Standardly, however, POST requests don't require a body and therefore don't require the Content-Length header.
I wrote a bash script that gets output from a website using curl and does a bunch of string manipulation on the html output. The problem is when I run it against a site that is returning its output gzipped. Going to the site in a browser works fine.
When I run curl by hand, I get gzipped output:
$ curl "http://example.com"
Here's the header from that particular site:
HTTP/1.1 200 OK
Server: nginx
Content-Type: text/html; charset=utf-8
X-Powered-By: PHP/5.2.17
Last-Modified: Sat, 03 Dec 2011 00:07:57 GMT
ETag: "6c38e1154f32dbd9ba211db8ad189b27"
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: must-revalidate
Content-Encoding: gzip
Content-Length: 7796
Date: Sat, 03 Dec 2011 00:46:22 GMT
X-Varnish: 1509870407 1509810501
Age: 504
Via: 1.1 varnish
Connection: keep-alive
X-Cache-Svr: p2137050.pubip.peer1.net
X-Cache: HIT
X-Cache-Hits: 425
I know the returned data is gzipped, because this returns html, as expected:
$ curl "http://example.com" | gunzip
I don't want to pipe the output through gunzip, because the script works as-is on other sites, and piping through gzip would break that functionality.
What I've tried
changing the user-agent (I tried the same string my browser sends, "Mozilla/4.0", etc)
man curl
google search
searching stackoverflow
Everything came up empty
Any ideas?
curl will automatically decompress the response if you set the --compressed flag:
curl --compressed "http://example.com"
--compressed
(HTTP) Request a compressed response using one of the algorithms libcurl supports, and save the uncompressed document. If this option is used and the server sends an unsupported encoding, curl will report an error.
gzip is most likely supported, but you can check this by running curl -V and looking for libz somewhere in the "Features" line:
$ curl -V
...
Protocols: ...
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
Note that it's really the website in question that is at fault here. If curl did not pass an Accept-Encoding: gzip request header, the server should not have sent a compressed response.
In the relevant bug report Raw compressed output when not using --compressed but server returns gzip data #2836 the developers says:
The server shouldn't send content-encoding: gzip without the client having signaled that it is acceptable.
Besides, when you don't use --compressed with curl, you tell the command line tool you rather store the exact stream (compressed or not). I don't see a curl bug here...
So if the server could be sending gzipped content, use --compressed to let curl decompress it automatically.