I am using NiFi API to build a custom application. I am unable to find an API that captures the processor history in a format as shown below. Does anyone know if this API exists?
I have tried many of the API's here, but it's not obvious any of these can do the trick.
NiFi REST Api documentation
You can use the GET/processors/{id} to get back this information in the form of a ProcessorEntity. This contains all fields as shown in your screenshot: name, type, status, counters of last 5 minutes...
Inside the ProcessorEntity object, you can find the ProcessorStatus object which contains the name, type, status, 5min snapshot information per node and aggregated etc...
I included some links as examples to these objects from the NiFi python client, but you can also find some examples on the NiFi REST API docs you linked yourself.
Related
I want to update my processor_group variable via a processor. Even ExecuteScript works. Can this be achieved?
You can only update the variable registry through the REST API, the same way the UI does.
It is a PUT to /process-groups/{id}/variable-registry:
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html
Just as an update to this Answer:
The nifi url you need is (default basic nifi node):
http://[IP ADDRESS]:8080/nifi-api/
The full api url you need to POST an update to an attribute is:
http://[IP ADDRESS]:8080/nifi-api/process-groups/[Process Group ID]/variable-registry/update-requests
An example of the payload is:
{"processGroupRevision":{"clientId":"c530ce77-0174-1000-fb36-93fa5e92e574","version":6},"disconnectedNodeAcknowledged":false,"variableRegistry":{"processGroupId":"c53cdaff-0174-1000-7922-5285dec53a94","variables":[{"variable":{"name":"authToken","value":"test3"}}]}}
To learn more about the payload, do some variable work in your NiFi UI and watch your developer tools for the "update-requests" entries and pay attention to the [Process Group ID] and the Payload.
I am trying to use the rest API to dynamically update and control my Apache NiFi Flow. I am using Postman to explore the REST API but am having trouble deleting properties/descriptors.
My current process is to call a GET to this address - http://localhost:8080/nifi-api/processors/{ID}
I then modify the response as desired and do a PUT with the modified response as the body. If I add a descriptor or change the content of a descriptor it works ok. But if I try to delete a descriptor by removing it from the properties and descriptors area then nothing happens.
I still get a 200 OK response, but it is the same as the original.
I am using NiFi 1.1.2 on Windows.
The PropertyDescriptors are specified by the Processor in question. These are read-only values and describe the properties the Processor currently supports. In you want to remove a property, and it is optional, you should be able to remove the value for it by setting it's entry to null in properties object in your request.
I see the is the ability to get all components of a project by doing
/project/[projectkey]/components
but I don't see any capability or documentation on how to get the list of labels that are available for a project (something like:
/project/[projectkey]/labels
Does the JIRA REST API support querying the list of labels available on a project?
Just to clarify, labels (at least the built-in JIRA ones) are global entities so they can be attached to any Issue in any Project.
As to your question - no, there's no public REST endpoint to get/change/add labels to JIRA.
Jira Cloud has /rest/api/3/label.
Jira Server provides /rest/api/2/jql/autocompletedata/suggestions?fieldName=labels which is however not paginated and only returns the first few labels (label values can be queried using &fieldValue=X).
However, as hacky workaround you can misuse the API endpoints some Atlassian Jira Gadgets are communicating with. Though this has the following disadvantages:
Are internal APIs
Atlassian apparently plans to replace Gadgets with Dashboard Items eventually
Might change behavior, see e.g. JRASERVER-67446
No pagination (?), responses can be huge
Responses are designed for Gadgets, therefore contain irrelevant data
Labels Gadget
/rest/gadget/1.0/labels/gadget/project-<PROJECT_ID>/labels
Where <PROJECT_ID> is the numeric ID of the project.
Heat Map Gadget
/rest/gadget/1.0/heatmap/generate?projectOrFilterId=<PROJECT_OR_FILTER>&statType=labels
Where <PROJECT_OR_FILTER> can be either:
project-<PROJECT_ID>
filter-<FILTER_ID>
I have build a custom connector to get the data from a web service and then index it. The web service response returns only the data to be indexed.
I want to delete the documents from index which are not part of the web service response during the crawl but were added to the index in the last crawl.
Is there any way to achieve the above or can I flush the full index programmatically in the connector code and then add the recent content to the index.
Marged is correct. A feed (which is what the connector can send to the GSA) of type full will purge the existing feed and replace it. Otherwise, your connector is going to have to manage state and prune out documents as you decided.
Thanks Marged and Michael for the help.. I guess i have to write the custom logic in connector to delete the data from index.
What you're trying to achieve is exactly what happens when you send a "full" content feed. This is from the documentation:
When the feedtype element is set to full for a content feed, the system deletes all the prior URLs that were associated with the data source. The new feed contents completely replace the prior feed contents. If the feed contains metadata, you must also provide content for each record; a full feed cannot push metadata alone. You can delete all documents in a data source by pushing an empty full feed.
Marged is correct that v4.x is the way to go in the future, but if you've already started this with the 3.x connector framework and you're happy with it there's no need to rush to upgrade it. All the related code is open source and 3.x won't disappear any time soon, there are too many 3rd party connectors based on it.
I'm trying to use the Google Analytics Core Reporting API and have the following troubles. I know how to fetch that I want the problem I have is:
I want to use a filter only for specific metrics and not for all of them.
Is this even possible?
Here is an example how my Reporting "Code" looks like:
Metrics "ga:sessions
ga:goal6Completions
ga:goal7Completions
ga:goal10Completions
ga:totalEvents
ga:uniqueEvents"
Dimensions "ga:date"
Filters "ga:eventCategory==customerType"
I have the same problems with segments as with the filters that all metrics all segmented/filtered and only specific metrics should.
There really isn't another way other to get the data you want without creating a new request where the filter isn't applied unfortunately.
Just remember that you are only allowed five requests total. To give context, this may only apply to how I use the Reporting API, but I use Google App Script to push data to Google Sheets.
This may or may not be helpful to you, but the reporting code I use I have made available in a repo on GitHub: https://github.com/jessfeliciano/aggregateGoogleAnalyticsReporting/blob/master/objectQueryWithFilter.js
I've included a guide that gives additional information regarding creating requests in general, so it may spark other ways to go about creating your data trees. I hope this helps.