My Input clause produces a value A and another value B, and I want to send an email notification with some text that contain the result of A/B in it. I had an early version that worked locally where I used a Groovy script in a Transform clause so I had a new variable ctx.payload.result holding the result of A/B, but elastic.co will only let me use Expression scripts because of security reasons. I also tried to resolve the expression inline in the email's body {{var}} tag, but apparently it doesn't resolve expressions.
Remember, I can't use Groovy to modify the payload. Any ideas?
Elastic's Cloud allows users to enable the Groovy scripts as well (disabled by default). From the cluster creation page of Cloud:
Elasticsearch can use scripts to implement flexible ranking, filtering, faceting and more. It is important to restrict their usage, as they enable arbitrary code execution. When enabling sandboxed scripts the Painless scripting language is enabled in 5.0. In the older versions "expression" and "mustache" languages will be enabled. When enabling all scripts, Groovy and any language provided by plugins will be available.
Related
I am trying to get the information of all the files related to an instance of ibm bpm but the following query does not work for me and there is no error in the javascript console either. I am using ECM Document List and in configuration I am adding a variable which contains the query.
"SELECT cmis:name, IBM_BPM_Document_FileNameURL,IBM_BPM_Document_UserId FROM IBM_ WHERE IBM_BPM_Document_ProcessInstanceId = 75774"
Thanks
I would assume that you are using and external ECM server and not the embedded ECM that comes along with IBM BPM/BAW.
Looking at the problem, I would approach the debugging in the following order;
Use a relevant ECM browser (ACCE in case of Filenet) to check if the
documents have a property that holds the value for the instanceID.
Because, by default external ECM servers don't have such a document
property.
If the document has such a property, then use that in the
"WHERE" clause of the query. If it doesn't then talk to whoever
maintains the ECM environment to create such a property and make
sure that is set properly (to the correct instanceId) for the
documents.
Another solution if you have access to it can be using the "BPM Document List" and "BPM File Uploader" which has the feature (as a configuration option) to associate documents with the current process instance.
Nifi custom properties (per-environment property files) looks to be a perfect way to define an environment specific paths and credentials. The only issue is how to keep sensitive information there? There is Nifi Encrypt-Config Tool described in more details here.
Is Nifi Encrypt-Config Tool capable of encrypting variable files (defined with nifi.variable.registry.properties) besides nifi.properties?
As far as I understood, it encrypts only nifi.properties. It's important, because with Nifi Docker Image I can define only nifi.variable.registry.properties (NIFI_VARIABLE_REGISTRY_PROPERTIES env var) without ability to modify nifi.properties.
The NiFi encrypt-config tool interacts with the following configuration files:
nifi.properties
login-identity-providers.xml
authorizers.xml
bootstrap.conf
flow.xml.gz
It does not handle any linked custom variable definition files, and there is no mechanism for sensitive variables to be properly secured and stored. Variables do not support any sensitive values at all for this reason.
Variables are treated as deprecated in modern versions of NiFi -- still supported but their use is discouraged -- and parameters were introduced in version 1.10.0 as a modern solution. Parameters do support sensitive values and are accessible from every property descriptor at the framework level rather than on a per-field basis depending on the developer's explicit decision to support them. You should prioritize parameters for the storage of sensitive values needed in your flow definitions.
Depending on your threat model, you may have less robust but acceptable alternatives:
If you accept the security level of environment variables, you can populate these directly and they will be referenced in any properties which support Expression Language, the same as "NiFi variables"
You can edit the nifi.properties file through a custom Docker image, startup scripts, etc. Any modified or added properties in that file can be encrypted by adding their key (property key descriptor, not cryptographic key) as a comma-delimited list to nifi.sensitive.props.additional.keys in that file. These properties will also be protected by the toolkit and decrypted in memory during NiFi application startup. However, nifi.properties is meant to hold framework-level configuration values, not component-level properties.
The whole morning I have been trying to setup e2e tests reporting via SonarQube's Generic Execution, by using the Generic Test Data -> Generic Execution feature.
I created a custom xml report that gets added to the scan properties like this:
sonar.testExecutionReportPaths=**/e2e-report.xml
So far, SonarQube seems to completely ignore this property and I no attempt to parse the file in the logs. Has anyone made it work?
These are links by Sonar about the Generic Execution feature:
https://docs.sonarqube.org/display/SONAR/Generic+Test+Data
https://github.com/SonarSource/sonarqube/blob/master/sonar-scanner-engine/src/main/java/org/sonar/scanner/genericcoverage/GenericTestExecutionSensor.java
This is a SonarQube 6.2+ feature. Make sure to use an appropriate SonarQube version.
In addition sonar.testExecutionReportPaths does not allow matchers (like *).
Please provide relative or absolute paths, comma separated.
See also:
The official documentation of the Generic Test Data feature
The source code, that looks up the generic execution files
I have NiFi template exported as xml. I am using rest-api to upload template to a NiFi instance. Now, I want to update/add some properties (say, password) of the template from rest-api (or any other option available, programatically).
I read the docs and various community threads without success. Referred links:
How to set props of processor
Update nifi flow on the fly
Open for any approach,
Thanks
I think there is a bit of confusion in your wording. Correct me if I'm wrong but I believe what you want to do is:
Create a template in one location
Export it
Upload it to another NiFi instance
Add the template to the canvas (so now it's just components on your NiFi canvas)
Edit the properties of the components that were added
There are generally two different reasons you would want to edit the properties after importing a template: the properties are specific to the instance you're running on; they were sensitive properties.
With the addition of the "variable registry" in NiFi-0.7.0 you can have multiple files that at NiFi's start-up are read in to give custom variables to use. Here is a section about it in the NiFi docs. This allows you to have custom variables to reference via Expression Language (EL) specific to each environment you run on.
The "variable registry" doesn't help for the sensitive properties though, because the EL used to reference them doesn't get exported with the template (since the property is sensitive). You will need to use the rest-api to update the processor properties explicitly. The NiFi docs give the exact call to use to update a processor (under Processors -> Put). Upgrading the variable registry to work securely is on the NiFi roadmap.
If I was completely off and you simply want to modify a template after importing it into a NiFi instance. You would have to add the template to your graph, delete the template from the listing, re-create it using the components on your graph. After templates are imported/created they are immutable.
I've been scouring through the rsyslog documentation for a way to anonymize mysql log data by removed quoted strings. I've successfully managed to detect strings with sensitive data using the :contains property but I can't seem to find a way to replace.
I've looked through the property options and the regex functionality. I believe I may be missing something because none of those provide a straight way for find and replace.
AFAIK, there's no way currently to do regex replace in rsyslog. The cleanest way (I see) for achieving what you need is to parse your logs with mmnormalize (more documentation can be found at liblognorm, which is the library mmnormalize uses). Then, you can access all the parsed properties, and put whatever you want in templates. Templates let you select what properties from the messages get written in MySQL.
The benefit of this solution is that mmnormalize should be faster than using regular expressions. The problem is that you'll probably need a new version of rsyslog (probably 8.x) to get it working properly.