I am trying to use the rest API to dynamically update and control my Apache NiFi Flow. I am using Postman to explore the REST API but am having trouble deleting properties/descriptors.
My current process is to call a GET to this address - http://localhost:8080/nifi-api/processors/{ID}
I then modify the response as desired and do a PUT with the modified response as the body. If I add a descriptor or change the content of a descriptor it works ok. But if I try to delete a descriptor by removing it from the properties and descriptors area then nothing happens.
I still get a 200 OK response, but it is the same as the original.
I am using NiFi 1.1.2 on Windows.
The PropertyDescriptors are specified by the Processor in question. These are read-only values and describe the properties the Processor currently supports. In you want to remove a property, and it is optional, you should be able to remove the value for it by setting it's entry to null in properties object in your request.
Related
I am using NiFi API to build a custom application. I am unable to find an API that captures the processor history in a format as shown below. Does anyone know if this API exists?
I have tried many of the API's here, but it's not obvious any of these can do the trick.
NiFi REST Api documentation
You can use the GET/processors/{id} to get back this information in the form of a ProcessorEntity. This contains all fields as shown in your screenshot: name, type, status, counters of last 5 minutes...
Inside the ProcessorEntity object, you can find the ProcessorStatus object which contains the name, type, status, 5min snapshot information per node and aggregated etc...
I included some links as examples to these objects from the NiFi python client, but you can also find some examples on the NiFi REST API docs you linked yourself.
I want to update my processor_group variable via a processor. Even ExecuteScript works. Can this be achieved?
You can only update the variable registry through the REST API, the same way the UI does.
It is a PUT to /process-groups/{id}/variable-registry:
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html
Just as an update to this Answer:
The nifi url you need is (default basic nifi node):
http://[IP ADDRESS]:8080/nifi-api/
The full api url you need to POST an update to an attribute is:
http://[IP ADDRESS]:8080/nifi-api/process-groups/[Process Group ID]/variable-registry/update-requests
An example of the payload is:
{"processGroupRevision":{"clientId":"c530ce77-0174-1000-fb36-93fa5e92e574","version":6},"disconnectedNodeAcknowledged":false,"variableRegistry":{"processGroupId":"c53cdaff-0174-1000-7922-5285dec53a94","variables":[{"variable":{"name":"authToken","value":"test3"}}]}}
To learn more about the payload, do some variable work in your NiFi UI and watch your developer tools for the "update-requests" entries and pay attention to the [Process Group ID] and the Payload.
I have build a custom connector to get the data from a web service and then index it. The web service response returns only the data to be indexed.
I want to delete the documents from index which are not part of the web service response during the crawl but were added to the index in the last crawl.
Is there any way to achieve the above or can I flush the full index programmatically in the connector code and then add the recent content to the index.
Marged is correct. A feed (which is what the connector can send to the GSA) of type full will purge the existing feed and replace it. Otherwise, your connector is going to have to manage state and prune out documents as you decided.
Thanks Marged and Michael for the help.. I guess i have to write the custom logic in connector to delete the data from index.
What you're trying to achieve is exactly what happens when you send a "full" content feed. This is from the documentation:
When the feedtype element is set to full for a content feed, the system deletes all the prior URLs that were associated with the data source. The new feed contents completely replace the prior feed contents. If the feed contains metadata, you must also provide content for each record; a full feed cannot push metadata alone. You can delete all documents in a data source by pushing an empty full feed.
Marged is correct that v4.x is the way to go in the future, but if you've already started this with the 3.x connector framework and you're happy with it there's no need to rush to upgrade it. All the related code is open source and 3.x won't disappear any time soon, there are too many 3rd party connectors based on it.
I am using TeamCity's REST api to get the list of changed files for a specific change.
I do it like this:
http://teamcityserver:8111/httpAuth/app/rest/changes/id:
In the result, I am getting a bunch of "file" xml nodes. However, for each file, I would like to know the action done on it (was this file added? removed? edited?) and this information is not present in the file nodes.
Is there a way I can get this information from the api?
(I am using TeamCity 9, if that's relevant)
I just got a response from TeamCity support saying that it's not possible:
"Unfortunately it is not possible to get the file action via REST API".
I have problem with Spring-DATA-REST. I allready posted in the spring forum but got no answer. Maybe here someone can help?
I have a test application set up with Spring-DATA-REST and all my domain objects get exposed by spring data rest.
Now: If I access my files repository under the uri /files/ I get a list of files I just created.
But: If I then access one of the files with the direct uri /files/{id}, for example /files/1
I get data from a file I entered long ago. I can't explain where this data comes from...
I just checked the JpaRepository. That is ok and I get the same data from a file retrieved with:
JpaRepository.findAll().get(0) as from
jpaRepository.findOne(JpaRepository.getAll().get(0).getId())
Is there some cache in spring data rest, that could still have the old data?
I am really desperate. Anyone has an idea on how to get my new data with spring data rest?
I assume that you are using HTTP as a transfer layer. If yes then you should look at HTTP cache headers in response. The result may be cached somewhere between.
You may use:
curl -vvl uri/files/{id}
to see all response's headers.