Can someone please suggest how to extract a portion of the absolute path in Nifi using the UpdateAttribute processor?
For instance, if I have an absolute path like:
C:\Users\ram\nifi\E7GW\TC02A\10R140\LC3P-7A511-AE\TempId-01
I need to extract the codes E7GW , TC02A & 10R140.
In updateAttribute, I created a property and assigned below value to it: ${absolute.path:getDelimitedField(12,'/')}
But this is just returning the formula that is used. Can someone please help?
Related
I have a GetFile Processor parsing some logs using a ExtractGrok Processor. Unfortunately the actual log files themselves contain the same naming convention request.log but the path to the logs is different. For example /var/log/server1/account1/request.log or /var/log/server1/account2/request.log. What I'd like to do is capture anything in /server1/ and /account1/ and store them as fields "host" and "account" so I can further partition the records.
I imagine this will look something like GetFile Processor ---> ExtractGrok Processor and then possibly PartitionRecord? But this is where I'm stuck. Partition Record would require the directory path to be part of the attributes so I can't include it. I guess I would need to somehow extract the file path before the PartitionRecord? I'm just not sure exactly where to do this.
GetFile should already add the filename and path attributes to the FlowFiles (see the documentation). You could use UpdateAttribute to get the values "server1" and "account1" out of the path attribute using Expression Language (see the getDelimitedField function for example).
To add the attribute(s) as fields you can use UpdateRecord with a GrokReader and then you don't need the ExtractGrok processor.
It doesn't sound like you need PartitionRecord as I'm presuming each FlowFile's records contain the same value for "host" and/or "account". If that's the case, and you don't need the "host" and "account" fields for anything else, you probably don't need all of the above components and could use UpdateAttribute -> RouteOnAttribute. If that's not the case, then you can use PartitionRecord on whichever other field(s) you want to partition on.
New to NiFi, struggling with some basics and would be grateful for any assistance...
I need to get files from a remote server and store them on a NiFi server. Using GetSFTP and PutFile this works ok. But I wish to keep the relative paths.
ie. on remote server /data/hosts/host01/.... would be copied into /imports/host01/... on NiFi
I have this working by using /imports/$path:substring(11)} as the Directory value in PutFile. I have tried using a parameter and a variable to store the Remote Path value and tried using the :length(var1) function of the Parameter and/or Variable within the the above substring function but nothing seems to work other than hard coding the substring length.
Is their an easier way to keep relative paths with get and put processors??
GetFile and PutFile require absolute paths for the directories. If you know the relationship between the root directory and the destinations, you can construct the full path using Expression Language.
<div><a src="What I need" data-src="What I don't need">Demo</a></div>
I am tried this xpath("./div/a/#src"),but it will give me all of that, but I don't want the #data-src, how should I do???
the raw page is here:
the raw page
first of all I would recommend you to change the #src-data in data-src!
it will avoid you many problems when parsing XML files.
Then you can use directly the following XPATH to get your src attribute:
/div/a/#src
if you make it start with . it will only access the relative path from the current node.
I am trying to use an environment variable which contains the file path, in NiFi Expression language. When I just give environment variable ${MY_VAR} for the file name, it is not getting resolved.
But if there is a space or some other character before the NiFi EL, it is being evaluated, but that will make an invalid file name. How to have this resolved without any extra characters.
So as per observation, when the whole content is just an EL it is not getting evaluated otherwise, it is.
PS: I am trying give the file name in Truststore Filename field in default StandardSSLContextService
You are encountering a bug in Apache NiFi. As Bryan pointed out, the truststore location field is not intended to support the Expression Language. The problem is stemming from an incorrect line at StandardSSLContextService:183 where the .evaluateExpressionAttributes() method is being invoked in the custom file validator, despite the comment a few lines above explaining the entire reason for this custom validator is because the default supports EL.
I ran a remote debugger and was able to evaluate ${TRUSTSTORE_LOCATION} using the environment variable TRUSTSTORE_LOCATION (which must be set before NiFi is started, because the VariableRegistry.ENVIRONMENT_SYSTEM_REGISTRY is only created by NotificationServiceManager which is invoked in RunNiFi:270). I was able to watch it correctly evaluate the environment variable to a file path, but then it was concatenated to the current running directory (.../nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/TRUSTSTORE_LOCATION) rather than being treated as an absolute/relative path because the property descriptor is unaware that this is the result of EL evaluation.
I have filed NIFI-4274 to document this bug. A workaround is to use a hard-coded truststore location in NiFi and use symlinks on your filesystem to point that at the actual file location if you do not know it at configure time.
I want to store a path like c:\abhor\test in a variable but it is treated as \a,\t. I don't want to use c:\\abhor\\test. Please provide some solution to this problem?