I have NiFi template exported as xml. I am using rest-api to upload template to a NiFi instance. Now, I want to update/add some properties (say, password) of the template from rest-api (or any other option available, programatically).
I read the docs and various community threads without success. Referred links:
How to set props of processor
Update nifi flow on the fly
Open for any approach,
Thanks
I think there is a bit of confusion in your wording. Correct me if I'm wrong but I believe what you want to do is:
Create a template in one location
Export it
Upload it to another NiFi instance
Add the template to the canvas (so now it's just components on your NiFi canvas)
Edit the properties of the components that were added
There are generally two different reasons you would want to edit the properties after importing a template: the properties are specific to the instance you're running on; they were sensitive properties.
With the addition of the "variable registry" in NiFi-0.7.0 you can have multiple files that at NiFi's start-up are read in to give custom variables to use. Here is a section about it in the NiFi docs. This allows you to have custom variables to reference via Expression Language (EL) specific to each environment you run on.
The "variable registry" doesn't help for the sensitive properties though, because the EL used to reference them doesn't get exported with the template (since the property is sensitive). You will need to use the rest-api to update the processor properties explicitly. The NiFi docs give the exact call to use to update a processor (under Processors -> Put). Upgrading the variable registry to work securely is on the NiFi roadmap.
If I was completely off and you simply want to modify a template after importing it into a NiFi instance. You would have to add the template to your graph, delete the template from the listing, re-create it using the components on your graph. After templates are imported/created they are immutable.
Related
If I created a NiFi flow template in NiFi v1.11.1, what is the best way to update that template flow for a newer NiFi build version say v1.15.0?
The XML template file has several processors referencing v1.11.1 and when loading it in v1.15.0, it is sometimes unable to load.
Do I need to re-construct this flow template in the newer NiFi build or should I manually modify the XML template file?
You do not need to start manually changing the template XML.
Update NiFi and your flow will update with it.
If things have changed, e.g. a processor has a new, required property, it will be left in an Invalid state with a warning tip that tells you that you need to add the missing property.
I am trying to get the information of all the files related to an instance of ibm bpm but the following query does not work for me and there is no error in the javascript console either. I am using ECM Document List and in configuration I am adding a variable which contains the query.
"SELECT cmis:name, IBM_BPM_Document_FileNameURL,IBM_BPM_Document_UserId FROM IBM_ WHERE IBM_BPM_Document_ProcessInstanceId = 75774"
Thanks
I would assume that you are using and external ECM server and not the embedded ECM that comes along with IBM BPM/BAW.
Looking at the problem, I would approach the debugging in the following order;
Use a relevant ECM browser (ACCE in case of Filenet) to check if the
documents have a property that holds the value for the instanceID.
Because, by default external ECM servers don't have such a document
property.
If the document has such a property, then use that in the
"WHERE" clause of the query. If it doesn't then talk to whoever
maintains the ECM environment to create such a property and make
sure that is set properly (to the correct instanceId) for the
documents.
Another solution if you have access to it can be using the "BPM Document List" and "BPM File Uploader" which has the feature (as a configuration option) to associate documents with the current process instance.
Nifi custom properties (per-environment property files) looks to be a perfect way to define an environment specific paths and credentials. The only issue is how to keep sensitive information there? There is Nifi Encrypt-Config Tool described in more details here.
Is Nifi Encrypt-Config Tool capable of encrypting variable files (defined with nifi.variable.registry.properties) besides nifi.properties?
As far as I understood, it encrypts only nifi.properties. It's important, because with Nifi Docker Image I can define only nifi.variable.registry.properties (NIFI_VARIABLE_REGISTRY_PROPERTIES env var) without ability to modify nifi.properties.
The NiFi encrypt-config tool interacts with the following configuration files:
nifi.properties
login-identity-providers.xml
authorizers.xml
bootstrap.conf
flow.xml.gz
It does not handle any linked custom variable definition files, and there is no mechanism for sensitive variables to be properly secured and stored. Variables do not support any sensitive values at all for this reason.
Variables are treated as deprecated in modern versions of NiFi -- still supported but their use is discouraged -- and parameters were introduced in version 1.10.0 as a modern solution. Parameters do support sensitive values and are accessible from every property descriptor at the framework level rather than on a per-field basis depending on the developer's explicit decision to support them. You should prioritize parameters for the storage of sensitive values needed in your flow definitions.
Depending on your threat model, you may have less robust but acceptable alternatives:
If you accept the security level of environment variables, you can populate these directly and they will be referenced in any properties which support Expression Language, the same as "NiFi variables"
You can edit the nifi.properties file through a custom Docker image, startup scripts, etc. Any modified or added properties in that file can be encrypted by adding their key (property key descriptor, not cryptographic key) as a comma-delimited list to nifi.sensitive.props.additional.keys in that file. These properties will also be protected by the toolkit and decrypted in memory during NiFi application startup. However, nifi.properties is meant to hold framework-level configuration values, not component-level properties.
I'm working with Filenet P8 Content Platform Engine 5.2.1 in my current project. I wanna export classes, property templates, folder hierarchy and other assets from specific Filenet object store (there are three object stores on server) via Filenet Deployment Manager. When i'm try to retrieve object store data, there are all existing object stores are retrieved. How can i retrieve only specific object store via Filenet Deployment Manager? How should i config my environment for it?
When you "Retrieve Data for Object Store Half Map" you are given a choice of the "Source For Object Store Data".
If you choose "Deploy Data Set File" you will be able to get data from a single OS.
What I do is create a "Security" document. It is a record with all the security information I will need for FDM on a content-less document object within my desired OS.
I then create a Export Manifest, adding that record with no Include Options.
Export that document.
Then run the Retrieval for OS, Security, Service, and Connection Points.
You may want to familiarize yourself with the FDM instructions.
When you create a mapping and load the object store, it creates a xml file which will have list of objectstores. Try modifying the xml and see if it works.
PS: I have not tried this solution.
Regards,
Manju
I'm following the steps from the Adobe instructions on How to Build AEM Projects using Maven and I'm not seeing how to populate or configure the meta data for the contents.
I can edit and configure the actual files, but when I push the zip file to the CQ instance, the installed component has a jcr:primaryType of nt:folder and the item I'm trying to duplicate has a jcr:primaryType of cq:Component (as well as many other properties). So is there a way to populate that data without needing to manual interact with CQ?
I'm very new to AEM, so it's entirely possible I've overlooked something very simple.
Yes, this is possible to configure JCR node types without manually changing with CQ.
Make sure you have .content.xml file in component folder and it contains correct jcr:primaryType ( e.g. jcr:primaryType="cq:Component").
This file contains metadata for mapping JCR node on File System.
For beginners it may be useful take a look VLT, that used for import/export JCR on File System. Actually component's files in your project should be similar to VLT component export result from JCR.