Apache Nifi windows unable to load NAR library bundles - windows

I'm only attempting to launch the Nifi UI as a local instance to start playing with it. I've unzipped the package and made sure to set the JAVA_HOME variable to my Java 1.8. When I try to bin/run-nifi, in my nifi-app log, the error message is:
2018-05-03 15:03:50,585 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2018-05-03 15:03:52,330 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be 'Z:\DoE\LOCAL-~1\NIFI-1~1.0\.\conf\nifi.properties'
2018-05-03 15:03:52,363 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 146 properties from Z:\DoE\LOCAL-~1\NIFI-1~1.0\.\conf\nifi.properties
2018-05-03 15:03:52,423 INFO [main] org.apache.nifi.NiFi Loaded 146 properties
2018-05-03 15:03:52,779 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 64802
2018-05-03 15:03:53,071 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2018-05-03 15:03:53,181 WARN [main] org.apache.nifi.nar.NarUnpacker Unable to load NAR library bundles due to java.io.IOException: Z:\DoE\LOCAL-~1\NIFI-1~1.0\.\work\nar\framework directory does not have read/write privilege Will proceed without loading any further Nar bundles
2018-05-03 15:03:53,242 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.io.IOException: Z:\DoE\LOCAL-~1\NIFI-1~1.0\.\work\nar\framework could not be created
java.io.IOException: Z:\DoE\LOCAL-~1\NIFI-1~1.0\.\work\nar\framework could not be created
at org.apache.nifi.util.FileUtils.ensureDirectoryExistAndCanReadAndWrite(FileUtils.java:48)
at org.apache.nifi.nar.NarClassLoaders.load(NarClassLoaders.java:155)
at org.apache.nifi.nar.NarClassLoaders.init(NarClassLoaders.java:131)
at org.apache.nifi.NiFi.<init>(NiFi.java:133)
at org.apache.nifi.NiFi.<init>(NiFi.java:71)
at org.apache.nifi.NiFi.main(NiFi.java:292)
2018-05-03 15:03:53,383 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2018-05-03 15:03:53,387 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
I've followed the installation instructions and haven't been able to trouble shoot. How do I load these NAR files upon running Nifi?
Thanks

I believe the underlying error in your output is java.io.IOException: Z:\DoE\LOCAL-~1\NIFI-1~1.0\.\work\nar\framework could not be created.
NiFi requires file permissions to create and write several directories, there is a list in the NiFi Admin Guide: How to install and start NiFi. NiFi does this to unpack the NAR files, write logs, and for various data repositories that comprise your data flow.
You have a few options:
Modify the permissions of the directory to allow NiFi read/write access. This can be done for each individual child directory.
Copy the entire NiFi distribution to a read/write location and run it from there.
Edit the conf/nifi-properties file to change the locations of these directories to read/write locations. See NiFi Admin Guide: System Properties for help on the properties.
Symlinks are a great solution for systems that support symlinks.

Two things you can try:
Run NiFi with administrator privilege (not a good practice) by going to ~\<NIFI_INSTALLATION_DIR>\bin and right click run-nifi.bat. Click Run as Administrator
Move NiFi directory to a location where the logged in user has full access to. Ex: C:\Users\<YOUR_USER>\Documents\. Now try to execute bin\run-nifi.bat

Similarly to the resolution that James proposed. I had to do the below 3-step process.
My scenario: I'm using docker containers and had the same problem. Even changing the user of my container to root didn't work. So, I did the following:
1 - Download Minifi https://nifi.apache.org/minifi/download.html
2 - Untar and execute the Minifi agent on my own laptop (I'm using MAC) so that the necessary folders and files will be created.
3 - Tar it up again and add to the DockerFile of my container creation
Done! Everything worked fine after that.

Related

I can't start Apache Nifi

When I run run-nifi.bat it pops up for a split second but then auto closes. I don't really understand this, I just need it for a university class and it hadn't been properly explained, so I'm just trying on my own really.
I get this in my nifi-app.log:
2021-05-29 17:07:30,179 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2021-05-29 17:07:30,450 INFO [main] org.apache.nifi.security.kms.CryptoUtils Determined default nifi.properties path to be 'D:\SYSTEM\Downloads\nifi-1.13.2-bin\nifi-1.13.2\.\conf\nifi.properties'
2021-05-29 17:07:30,454 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 188 properties from D:\SYSTEM\Downloads\nifi-1.13.2-bin\nifi-1.13.2\.\conf\nifi.properties
2021-05-29 17:07:30,465 INFO [main] org.apache.nifi.NiFi Loaded 188 properties
2021-05-29 17:07:30,705 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 63487
2021-05-29 17:07:30,711 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.net.ConnectException: Connection refused: connect
java.net.ConnectException: Connection refused: connect
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:576)
at java.base/sun.nio.ch.Net.connect(Net.java:565)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:333)
at java.base/java.net.Socket.connect(Socket.java:645)
at java.base/java.net.Socket.connect(Socket.java:595)
at org.apache.nifi.BootstrapListener.sendCommand(BootstrapListener.java:102)
at org.apache.nifi.BootstrapListener.start(BootstrapListener.java:74)
at org.apache.nifi.NiFi.<init>(NiFi.java:102)
at org.apache.nifi.NiFi.<init>(NiFi.java:71)
at org.apache.nifi.NiFi.main(NiFi.java:303)
2021-05-29 17:07:30,712 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2021-05-29 17:07:30,712 INFO [Thread-0] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
I've tried editing the web properties in the config files in case the default was wrong. Right now it's on, but the errors are the same:
nifi.web.http.host=localhost
nifi.web.http.port=9090
nifi.web.http.network.interface.default=
I have Windows 10 Home Edition.
NiFi requires Java 8 or Java 11 to run. So your environment variables should point to the correct directory with Java 8 or Java 11.
Have you tried setting the JAVA_HOME environment variable? I would recommend checking the config files and telling the configs where to find the Java installation
You might be missing URL ACL
Maybe you can try below command:
netsh http add urlacl url=http://computername:port/ user=username
Source: https://serverfault.com/a/246798/191420

Cannot run SonarQube as Service

If I just run the StartSonar.bat everything works and starts fine.
If I setup a service to run using NSSM using:
Path: ..\blah\wrapper.exe
Startup directory: ..\blah
Arguments: -c ..\blah\conf\wrapper.conf
I get an error in the logs that says:
2019.06.24 16:03:49 INFO web[][o.s.p.ProcessEntryPoint] Starting web
2019.06.24 16:03:50 INFO web[][o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read
2019.06.24 16:03:51 INFO web[][o.e.p.PluginsService] no modules loaded
2019.06.24 16:03:51 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.join.ParentJoinPlugin]
2019.06.24 16:03:51 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]
2019.06.24 16:03:51 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
2019.06.24 16:03:52 INFO web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [127.0.0.1:9001]
2019.06.24 16:03:52 INFO web[][o.s.s.p.LogServerVersion] SonarQube Server / 7.7.0.23042 / 1dcac8b8de36b377a1810cc8f1c4c31744e12729
2019.06.24 16:03:52 INFO web[][o.sonar.db.Database] Create JDBC data source for jdbc:sqlserver://BRKPRCSQUBE1;databaseName=sonar;integratedSecurity=true
2019.06.24 16:03:53 ERROR web[][o.s.s.p.Platform] Web server startup failed
java.lang.IllegalStateException: Can not connect to database. Please check
connectivity and settings (see the properties prefixed by 'sonar.jdbc.').
That leads me to believe there is a problem with setting up the database correctly, but once again.. if I run the StartSonar.bat everything works fine.
The only thing I can think of is that I am setting up the service as a "Local System account," instead of when I run the bat it would be from my own account.
That being said, I cannot use my account and then login and change my password or something every time my AD changes.

Apache NiFi closes immediately with error, org.apache.nifi.StdErr Failed to start web server

I have recently installed NiFi, and it was working fine for few days. But suddenly today when i try to open it using run-nifi.bat, the NiFi window is getting closed in few seconds stating the below error:
2019-04-11 23:07:40,146 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read status file C:\Users\DOWNLO~1\NIFI-1~1.1-B\NIFI-1~1.1\bin\..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2019-04-11 23:07:40,149 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 54149
2019-04-11 23:08:00,352 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: Must configure HTTP or HTTPS connector
2019-04-11 23:08:00,352 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2019-04-11 23:08:00,419 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
I do looked out for the org.apache.nifi.StdErr Failed to start web server: Must configure HTTP or HTTPS connector error, but unfortunately I cant find a similar one. I'm sure that no settings or properties has been changed since installation. Any suggestion guys?
I was getting the same error.
You need to check nifi-app.log file to get more details on this type of error.
Here’s what I did: Remove the port information from nifi.properties for HTTPS and only keep the setting for HTTP. Then restart nifi again.
Keep one property enable like https or http. in my case https:127.0.0.1:8443/nifi/
it works fine me.

Apache Nifi failed to start web server (shutting down) during DocGenerator.generate()

Apache-Nifi v-1.7.1 was running fine on AWS Centos machine but now on restart it thrown following error: (below content of /nifi-app.log)
org.apache.nifi:nifi-standard-nar:1.7.1 || ./work/nar/extensions/nifi-standard-nar-1.7.1.nar-unpacked
org.apache.nifi.processors.standard.FetchSFTP
org.apache.nifi:nifi-standard-nar:1.7.1 || ./work/nar/extensions/nifi-standard-nar-1.7.1.nar-unpacked
org.apache.nifi.processors.azure.eventhub.ConsumeAzureEventHub
org.apache.nifi:nifi-azure-nar:1.7.1 || ./work/nar/extensions/nifi-azure-nar-1.7.1.nar-unpacked
=== End Processor types ===
2019-01-21 10:07:15,987 WARN [main] org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
java.lang.NullPointerException: null
at org.apache.nifi.documentation.DocGenerator.generate(DocGenerator.java:62)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:835)
at org.apache.nifi.NiFi.<init>(NiFi.java:157)
at org.apache.nifi.NiFi.<init>(NiFi.java:71)
at org.apache.nifi.NiFi.main(NiFi.java:292)
2019-01-21 10:07:15,987 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2019-01-21 10:07:15,988 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
I ran into this issue when the folders pointed to by nifi.nar.library.directory.* was inaccessible, e.g. the Symbolic links were no longer valid due to file system changes.
Fixed the links and the server came up.
sudo chown -R nifi:nifi /opt/nifi/
sudo service nifi start
worked for me.

NiFi - java.lang.NoSuchMethodError

I am seeing some weird error in my nifi cluster, i have a 3node secured nifi cluster and we have some custom library nars when we deploying in all nodes 2 nodes starting up without any issues but in one node i am seeing the below errors.
2017-04-20 23:43:34,516 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.NoSuchMethodError: org.apache.nifi.processor.util.StandardValidators.createControllerServiceExistsValidator(Ljava/lang/Class;)Lorg/apache/nifi/components/Validator;
java.lang.NoSuchMethodError: org.apache.nifi.processor.util.StandardValidators.createControllerServiceExistsValidator(Ljava/lang/Class;)Lorg/apache/nifi/components/Validator;
Did you try deleting nar/ folder under work/ directory ,add your nars to lib/ and restart NiFi ? Usually, NiFi expands the nar bundles during start up.
Also, are you extending any standard processors to build a custom NiFi processor ? In such case you might have to add a nar dependency.

Resources