If I created a NiFi flow template in NiFi v1.11.1, what is the best way to update that template flow for a newer NiFi build version say v1.15.0?
The XML template file has several processors referencing v1.11.1 and when loading it in v1.15.0, it is sometimes unable to load.
Do I need to re-construct this flow template in the newer NiFi build or should I manually modify the XML template file?
You do not need to start manually changing the template XML.
Update NiFi and your flow will update with it.
If things have changed, e.g. a processor has a new, required property, it will be left in an Invalid state with a warning tip that tells you that you need to add the missing property.
Related
I want to add 3 extension to nifi (nifi-encryptMD5-nar-1.0.nar-unpacked,nifi-getOperator-nar-1.0-SNAPSHOT.nar-unpacked,nifi-splitAttributeValue-nar-1.0.nar-unpacked)
I added the extensions folder in the directory /opt/nifi/nifi-1.9.2/work/nar/extensions/
then when I restart the nifi service, nifi turns off and does not turn on, when I force the start with the user nifi, nifi turns on but the extentions have been deleted from the directory /opt/nifi/nifi-1.9.2/work/nar/extensions/
you have to put *.nar packages into nifi/lib directory.
nifi will extract it automatically on startup into nifi/work folder.
As daggett says, you need to use the .nar files, not any unpacked directories.
In your nifi.properties there will be two or more properties that provide locations for NiFi libraries:
nifi.nar.library.directory=./lib
nifi.nar.library.autoload.directory=./extensions
nifi.nar.library.directory.<something>=./<yourdir>
The first is the default and contains all the basic NiFi files. It is only checked on startup and any valid nars found are unpacked in the work directory and loaded. Generally you don't want to add anything here except in test environments as it complicates upgrades.
The second is empty by default but it is scanned every 30 seconds for new .nars. These will be unpacked and loaded if possible, but only for new libraries. Already loaded libraries will not be reloaded.
This is a good location to add your validated custom libraries without having to restart NiFi.
The third and further need to be added manually to the properties file. These are loaded on startup only and useful if you have a lot of custom processors and want to keep them organized.
In your situation I'd put the .nars in the extensions folder and check the logs to see if they were loaded successfully. You'll then need a full refresh of the browser window (Shift+F5 I think) before they show up in the list of processors.
In a cluster setup, add the .nars on all nodes and verify their availability before trying to add them to the canvas or things might get messy.
Good day,
I'm trying to upload nifi template and getting this kind of error:
org.apache.nifi.processors.kite.ConvertAvroSchema is not known to this NiFi instance
I suppose that my Nifi Instance misses some component - but where can I get it and where on file system I need to add missing file?
I'm little bit confused.
Thanks!
As stated here, since 1.10.0 kite-nar was taken out of the default binaries because of space limitations.
You can get it here.
How to clear the cache of FetchDistributedMapCache processor in Apache NiFi?
I tried deleting the persisted directory and also tried giving a new directory all together but it still fetches old data. Thanks for your help.
You should be able to stop the DistributedMapCacheClient and DistributedMapCacheServer, then delete the existing DistributedMapCacheServer and create a new one with same port as the previous one, then start them back up.
Inside NiFi, you could create a new DistributedMapCacheServer and point your processor at that instead. Outside of NiFi, I've written a Groovy script where you can interact with the DistributedMapCacheServer from the command line. The API only allows you to remove entries you know about; in the upcoming NiFi 1.2.0 release, you will be able to remove entries using a regular expression for the keys (implemented in NIFI-3627). At that point I will update the Groovy script to enable that feature.
I have NiFi template exported as xml. I am using rest-api to upload template to a NiFi instance. Now, I want to update/add some properties (say, password) of the template from rest-api (or any other option available, programatically).
I read the docs and various community threads without success. Referred links:
How to set props of processor
Update nifi flow on the fly
Open for any approach,
Thanks
I think there is a bit of confusion in your wording. Correct me if I'm wrong but I believe what you want to do is:
Create a template in one location
Export it
Upload it to another NiFi instance
Add the template to the canvas (so now it's just components on your NiFi canvas)
Edit the properties of the components that were added
There are generally two different reasons you would want to edit the properties after importing a template: the properties are specific to the instance you're running on; they were sensitive properties.
With the addition of the "variable registry" in NiFi-0.7.0 you can have multiple files that at NiFi's start-up are read in to give custom variables to use. Here is a section about it in the NiFi docs. This allows you to have custom variables to reference via Expression Language (EL) specific to each environment you run on.
The "variable registry" doesn't help for the sensitive properties though, because the EL used to reference them doesn't get exported with the template (since the property is sensitive). You will need to use the rest-api to update the processor properties explicitly. The NiFi docs give the exact call to use to update a processor (under Processors -> Put). Upgrading the variable registry to work securely is on the NiFi roadmap.
If I was completely off and you simply want to modify a template after importing it into a NiFi instance. You would have to add the template to your graph, delete the template from the listing, re-create it using the components on your graph. After templates are imported/created they are immutable.
I'm following the steps from the Adobe instructions on How to Build AEM Projects using Maven and I'm not seeing how to populate or configure the meta data for the contents.
I can edit and configure the actual files, but when I push the zip file to the CQ instance, the installed component has a jcr:primaryType of nt:folder and the item I'm trying to duplicate has a jcr:primaryType of cq:Component (as well as many other properties). So is there a way to populate that data without needing to manual interact with CQ?
I'm very new to AEM, so it's entirely possible I've overlooked something very simple.
Yes, this is possible to configure JCR node types without manually changing with CQ.
Make sure you have .content.xml file in component folder and it contains correct jcr:primaryType ( e.g. jcr:primaryType="cq:Component").
This file contains metadata for mapping JCR node on File System.
For beginners it may be useful take a look VLT, that used for import/export JCR on File System. Actually component's files in your project should be similar to VLT component export result from JCR.