I'm attempting to collect details about all processors on two installed instances of Nifi, versions 1.19.0 and 1.20.0. I want to use Nifi to do this so I can perform some minor ETLs and save the resultant dataset in a database. Both instances are configured as standalone authentication with SSL.
However, when I configure the InvokeHTTP processor how I think it should be to aquire a token, I'm getting a 400 status code and a response.body of "The username and password must be specified." on both instances.
Here's the super basic flow:
Nifi Flow
Here's what the error looks like:
Attributes of failed response
Here's the current config of the InvokeHTTP processor on the 1.20.0 instance which we can focus on from here out since they are both responding similarly.
Postman - InvokeHTTP Config
When I run the request with Postman, I get the expected token in the response body. And I'm able to make subsequent requests (the above config was a response from https://nifi06:8443/nifi-api/processors/39870615-0186-1000-e7d7-c59fa621ca7d in Postman).
I've tried the following:
Adding dynamic attributes for username/password as in the above configuration.
I've added them as JSON to the preceding Generate FlowFile processor's custom text.
And I've also tried using the Request Username/Request Password properties of the InvokeHTTP processor.
All return with the same response.body of "The username and password must be specified."
This seems it should be really simple and I'm sure I've been staring at it too long to see what I'm missing. Can anyone tell me where I need to specify the username/password?
Thanks.
Related
Using Apache Nifi v1.19.1, I need to make an HTTP POST request via the InvokeHTTP processor in order to authenticate with an API before I can upload the FlowFile via another InvokeHTTP processor.
I've configured the username/password for the API in a Parameter Context and marked them (as you might expect) as "Sensitive Value".
Now, when I try to reference those parameters in the GenerateFlowFile processor's Custom Text property, I get the error:
'Custom Text' is invalid because the property 'Custom Text' cannot reference Parameter 'username' because the Sensitivity of the parameter does not match the Sensitivity of the property.
(And again for the password parameter.) I understand why this is, but it seems like an interesting limitation. At some point I need to be able to utilize that username/password in my flow.
Is there a solution to this?
I have a datasource that I can access remotely with a POST request (including an API key and user ID as data in the request).
Is there anyway to configure Prometheus to scrape this datasource? Preferably without any additional software.
I think not using scrape_config without a proxy (or additional software) unless the datasource is accessible using one of the service discovery targest listed in Prometheus configuration.
If you need to use a proxy, this should be straightforward.
You'd need to construct an out-of-band HTTP server defined as the proxy to accept proxied (GET) requests from Prometheus, possibly1 (!) injecting the API key and user ID, do whatever's necessary to respond to Prometheus' GET with a list of metrics in the expected exposition format returning these as the scrape response.
1 - unless you can wrangle your inputs as HTML params, or as basic_auth or authorization headers.
I'm having troubles authenticating NiFi to a SOAP WebService, using a NTLM Authentication.
I've using this custom processor:
https://github.com/peetkes/nifi-http-processor
I've created a simple flow, with a FlowFile that contains the XML request, passed to the CustomInvokeHTTP Processor.
The problem is that I obtain "Error 401 Unauthorized".
I tried to use the same credentials with SOAPUI and they work.
I attach the flow and the processor properties.
Is there something else I can try?
I don't really know what to do.
Thank you so much!
I have a NiFi flow which needs to post the data to some other server, before posting it needs to fetch a value from my internal server or Nifi Distributed cache.
And add that Value in to Http header before posting the data, also update when gets particular error code 401.
Below is the flow I am trying, all green processor I could do but stuck at getting the value and adding in invoke http header . What are other process I can make us to achieve this .
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.0/org.apache.nifi.processors.standard.InvokeHTTP/
according to documentation on http code 401 InvikeHTTP routes file to No Retry relation.
also flow file should contain some response attributes including invokehttp.status.code
so, you could use RouteOnAttribute processor to redirect flow file to your external token processor only when invokehttp.status.code=401.
I am new to JMetet and I am having a lot of difficulties in understanding how it works.
I created a TC to add an object to my system using Blaze meter. Then, I imported the TC in Jmeter.
This TC fails when it should not (at least thats what I think) because whenever I use the system it works correctly:
This is the thread group if you need it to help me:
Am I doing something wrong? AM I missing something?
IMPORTANT: Should I be able to see my object added to the system if the TC passes?
As per HTTP Status Code 403 Forbidden description:
The HTTP 403 Forbidden client error status response code indicates that the server understood the request but refuses to authorize it.
This status is similar to 401, but in this case, re-authenticating will make no difference. The access is permanently forbidden and tied to the application logic, such as insufficient rights to a resource.
If your script assumes authentication most probably it fails somewhere somehow due to missing or improperly working correlation, for example this eedd968fe... bit
looks utterly suspicious, most probably you need to replace it with a some form of dynamic parameter extracted from the previous request using a suitable JMeter Post-Processor
Normally the flow looks like:
Open login page
Identify and extract all dynamic parameters and save them into JMeter Variables
Send the parameters along with credentials in the 2nd request
Check out Using Regular Expressions to Extract Tokens and Session IDs to Variables article for example challenge and solution