I'll comment on the problem i'm having.
In the company where I work, they decided a few years ago to implement nifi.
The problem with this is that they implemented the complete HDF solution, something really unnecessary to use only nifi.
With luck the Ambari interface is used to restart the service
I am currently in the process of migrating this nifi that consists of 3 clustered nodes (something also unnecessary since only small tasks are performed and resources are wasted) to a dockerized nifi standalone.
This process was done previously with a nifi with a smaller flow successfully.
The procedure that was carried out was simply to export a template of the flow and to import it, adding in the nifi.properties the correct keys to be able to use the controller services.
I find myself trying to perform this same procedure without success.
The flow is imported correctly but I can not visualize the controllers services that I have loaded by adding the correct key to the nifi.properties.
Try extracting the key from the controller services for load manually that appears as "sensitive value" but I could not do it.
As 2nd option copy the flow.xml.gz of the server to be migrated, edit some fields with a text editor (such as file paths, drivers, etc) and apparently I'm better with this;
I had several errors, among which I had to add the drivers (jdbc, hive, etc). I'm correcting small problems little by little.
Is there really any correct way to migrate this?
Because for more than correctly entering the password in the nifi.properties file, I can not see the controller services? Does HDF have any other configuration files?
As an extra data, the version of dockerized nifi used is 1.2.0, the version of nifi that currently has HDF is 1.1.0. Apparently there are no major changes that could affect.
Thanks for the help!
The sensitive properties in the flow.xml.gz file are encrypted with a key derived from nifi.sensitive.props.key entry in nifi.properties. If this value is itself encrypted (i.e. it is of the form abcdef...||abcdef...), you will also need to copy the nifi.bootstrap.sensitive.key value from bootstrap.conf. With those two values (and using the same algorithm for KDF and encryption as specified in nifi.sensitive.props.algorithm in nifi.properties), your new NiFi instance should be able to decrypt the sensitive processor properties that are embedded in an encrypted form in flow.xml.gz.
Thanks for the help
I did not know this about the master key in the bootstrap.conf file
I have done the following:
- Export from the original nifi the flow, and import it into the new nifi.
- Edit indicating the keys in the nifi.properties and bootstrap.conf
- The service does not start, the logs announce the following
2019-06-06 17:28:21,807 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: []
2019-06-06 17:28:21,809 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED
2019-06-06 17:28:21,809 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED
2019-06-06 17:28:21,809 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED
2019-06-06 17:28:21,829 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2019-06-06 17:28:21,829 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /opt/nifi/nifi-1.2.0
2019-06-06 17:28:21,829 INFO [main] org.apache.nifi.bootstrap.Command Command: /docker-java-home/bin/java -classpath /opt/nifi/nifi-1.2.0/./conf:/opt/nifi/nifi-1.2.0/./lib/slf4j-api-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/jul-to-slf4j-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/logback-classic-1.2.3.jar:/opt/nifi/nifi-1.2.0/./lib/jcl-over-slf4j-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-api-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/javax.servlet-api-3.1.0.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-framework-api-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/log4j-over-slf4j-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-properties-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/jetty-schemas-3.1.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-runtime-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-nar-utils-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/logback-core-1.2.3.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/opt/nifi/nifi-1.2.0/./conf/nifi.properties -Dnifi.bootstrap.listen.port=40303 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/opt/nifi/nifi-1.2.0/logs org.apache.nifi.NiFi -K /opt/nifi/nifi-1.2.0/./conf/sensitive.key
2019-06-06 17:28:21,912 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 35
2019-06-06 17:28:22,814 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 35155
2019-06-06 17:28:41,948 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: org.jasypt.exceptions.EncryptionOperationNotPossibleException
2019-06-06 17:28:41,949 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2019-06-06 17:28:42,922 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
After this it seems the service keeps restarting so replace the nifi-framework-nar file with version 1.1.0 so that it does not have the correct jasypt library. (Remember with i having migrate nifi 1.1 to 1.2)
Nifi keeps restarting announcing the following error.
2019-06-06 18:25:18,143 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2019-06-06 18:25:18,208 INFO [main] org.apache.nifi.NiFi Now overwriting file in /opt/nifi/nifi-1.2.0/./conf/sensitive.key
2019-06-06 18:25:18,209 INFO [main] org.apache.nifi.NiFi Overwriting key material with pad: c957a7de1ce10bf7a2c1b8cc8907df203e13bfb497e75b81e6fcd8cfbcae449d
2019-06-06 18:25:18,209 INFO [main] org.apache.nifi.NiFi Removing/unlinking file: /opt/nifi/nifi-1.2.0/./conf/sensitive.key
2019-06-06 18:25:18,209 INFO [main] org.apache.nifi.NiFi Read property protection key from key file provided by bootstrap process
2019-06-06 18:25:18,405 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/opt/nifi/nifi-1.2.0/./conf/nifi.properties'
2019-06-06 18:25:18,411 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 124 properties from /opt/nifi/nifi-1.2.0/./conf/nifi.properties
2019-06-06 18:25:18,597 INFO [main] o.a.n.properties.ProtectedNiFiProperties There are 1 protected properties of 5 sensitive properties (100%)
2019-06-06 18:25:18,604 INFO [main] o.a.n.p.AESSensitivePropertyProvider AES Sensitive Property Provider decrypted a sensitive value successfully
2019-06-06 18:25:18,605 INFO [main] org.apache.nifi.NiFi Loaded 123 properties
2019-06-06 18:25:18,614 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 34369
2019-06-06 18:25:18,696 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2019-06-06 18:25:18,718 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding 71 NAR files with all processors...
2019-06-06 18:25:22,910 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading process took 4192250200 nanoseconds.
2019-06-06 18:25:23,177 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-jetty-bundle-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jetty-bundle-1.2.0.nar-unpacked]
2019-06-06 18:25:23,186 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/framework/nifi-framework-nar-1.1.0.2.1.0.0-165.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/framework/nifi-framework-nar-1.1.0.2.1.0.0-165.nar-unpacked]
2019-06-06 18:25:23,188 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ldap-iaa-providers-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ldap-iaa-providers-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,193 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-gcp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-gcp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,194 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kerberos-iaa-providers-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kerberos-iaa-providers-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,195 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-standard-services-api-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-services-api-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,196 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hl7-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hl7-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,197 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-datadog-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-datadog-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,198 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-solr-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-solr-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,199 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-websocket-services-api-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-websocket-services-api-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,201 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ccda-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ccda-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,202 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-amqp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-amqp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,207 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-aws-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-aws-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,207 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-dbcp-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-dbcp-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,209 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-email-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-email-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,210 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-cassandra-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-cassandra-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,211 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-registry-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-registry-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,212 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kafka-0-10-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kafka-0-10-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,212 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kafka-0-9-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kafka-0-9-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,215 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-media-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-media-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,216 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-websocket-processors-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-websocket-processors-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,217 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-avro-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-avro-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,218 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-scripting-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-scripting-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,218 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ssl-context-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ssl-context-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,218 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-couchbase-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-couchbase-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,219 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-social-media-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-social-media-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,220 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kafka-0-8-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kafka-0-8-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,220 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-stateful-analysis-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-stateful-analysis-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,221 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-site-to-site-reporting-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-site-to-site-reporting-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,222 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-poi-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-poi-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,223 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-tcp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-tcp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,223 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-mqtt-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-mqtt-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,224 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-enrich-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-enrich-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,226 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hadoop-libraries-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hadoop-libraries-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,227 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-html-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-html-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,227 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-evtx-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-evtx-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,229 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-elasticsearch-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-elasticsearch-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,229 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-snmp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-snmp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,230 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-websocket-services-jetty-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-websocket-services-jetty-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,232 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-flume-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-flume-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,232 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-update-attribute-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-update-attribute-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,233 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-cybersecurity-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-cybersecurity-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,233 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-spring-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-spring-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,234 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-azure-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-azure-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,235 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kite-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kite-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,236 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-distributed-cache-services-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-distributed-cache-services-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,236 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ignite-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ignite-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,237 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-windows-event-log-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-windows-event-log-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,237 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-lumberjack-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-lumberjack-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,237 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-beats-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-beats-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,238 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-http-context-map-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-http-context-map-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,240 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-standard-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,241 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-elasticsearch-5-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-elasticsearch-5-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,241 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-riemann-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-riemann-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,242 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-slack-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-slack-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,242 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hwx-schema-registry-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hwx-schema-registry-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,243 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hive-services-api-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hive-services-api-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,243 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-provenance-repository-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-provenance-repository-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,243 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-mongodb-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-mongodb-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,244 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-splunk-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-splunk-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,244 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-language-translation-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-language-translation-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,245 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-record-serialization-services-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-record-serialization-services-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,246 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ambari-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ambari-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,246 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-jms-cf-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jms-cf-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,247 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-cdc-mysql-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-cdc-mysql-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,247 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hbase-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hbase-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,249 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hbase_1_1_2-client-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hbase_1_1_2-client-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,250 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hadoop-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hadoop-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,251 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-parquet-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-parquet-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,251 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-jms-processors-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jms-processors-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,254 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hive-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hive-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,323 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.NoSuchMethodException: org.apache.nifi.web.server.JettyServer.(org.apache.nifi.util.NiFiProperties, java.util.Set)
2019-06-06 18:25:23,324 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2019-06-06 18:25:23,325 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
Delete the folder of the flowfiles but it still does not start.
I did the test of using the folder lib of my nifi 1.1 and the error is the following one: Error: Could not find or load main class org.apache.nifi.bootstrap.RunNiFi
Where are the files specified to be used in nifi?
Apparently for more than amount the flow in the right way, the operation of the processors continues to depend on the files nar with the libraries belonging to its version.
Thanks!
Related
In my Windows 10 environment, I installed Hadoop 2.10.1 While starting the nodemanager using .\start-yarn.cmd command, I am getting following error message. The DataNode, ResourceManager and NameNode started properly though.
I need help in resolving this issue.
Error Log :-
STARTUP_MSG: build = https://github.com/apache/hadoop -r 1827467c9a56f133025f28557bfc2c562d78e816; compiled by 'centos' on 2020-09-14T13:17Z
STARTUP_MSG: java = 1.8.0_292
************************************************************/
22/08/16 10:27:20 INFO nodemanager.NodeManager: Node Manager health check script is not available or doesn't have execute permission, so not starting the node health script runner.
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ContainerEventDispatcher
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$ApplicationEventDispatcher
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event.LocalizationEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl$LocalizationEventHandlerWrapper
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncherEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainersLauncher
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerSchedulerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerScheduler
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl
22/08/16 10:27:20 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for class org.apache.hadoop.yarn.server.nodemanager.NodeManager
22/08/16 10:27:20 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
22/08/16 10:27:20 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
22/08/16 10:27:20 INFO impl.MetricsSystemImpl: NodeManager metrics system started
22/08/16 10:27:20 INFO nodemanager.DirectoryCollection: Disk Validator: yarn.nodemanager.disk-validator is loaded.
22/08/16 10:27:20 INFO nodemanager.DirectoryCollection: Disk Validator: yarn.nodemanager.disk-validator is loaded.
22/08/16 10:27:20 ERROR nodemanager.NodeManager: Error starting NodeManager
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Ljava/lang/String;I)V
at org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode(NativeIO.java:521)
at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:501)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:554)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:532)
at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1233)
at org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:183)
at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:212)
at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:747)
at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:743)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:743)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.createDir(DirectoryCollection.java:603)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.createDir(DirectoryCollection.java:601)
at org.apache.hadoop.yarn.server.nodemanager.DirectoryCollection.createNonExistentDirs(DirectoryCollection.java:390)
at org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerService.serviceInit(LocalDirsHandlerService.java:229)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerService.serviceInit(NodeHealthCheckerService.java:59)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:468)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:878)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:946)
22/08/16 10:27:20 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics system...
22/08/16 10:27:20 INFO impl.MetricsSystemImpl: NodeManager metrics system stopped.
22/08/16 10:27:20 INFO impl.MetricsSystemImpl: NodeManager metrics system shutdown complete.
22/08/16 10:27:20 INFO nodemanager.NodeManager: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NodeManager at DESKTOP-TFTO962/192.168.0.101
************************************************************/
When I tried to connect to Nifi UI using http://localhost:8080/nifi, i am getting below error
org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:331)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:299)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:398)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:935)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:297)
2020-02-27 11:51:11,834 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2020-02-27 11:51:11,836 INFO [Thread-1] o.eclipse.jetty.server.AbstractConnector Stopped ServerConnector#355ee205{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-02-27 11:51:11,837 INFO [Thread-1] org.eclipse.jetty.server.session node 0 Stopped scavenging
Can anyone suggest what is the cause of this issue?
Nifi version- 1.9.2,installed on windows machine
Here is the nifi status logs,
12:33:16.886 [main] DEBUG org.apache.nifi.bootstrap.NotificationServiceManager - Found 0 service elements
12:33:16.896 [main] INFO org.apache.nifi.bootstrap.NotificationServiceManager - Successfully loaded the following 0 services: []
12:33:16.897 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_STARTED
12:33:16.897 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_STOPPED
12:33:16.898 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_DIED
12:33:16.899 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File:
12:33:16.900 [main] DEBUG org.apache.nifi.bootstrap.Command - Properties: {pid=9724}
Failed to determine if Process 9724 is running; assuming that it is not
12:33:16.902 [main] INFO org.apache.nifi.bootstrap.Command - Apache NiFi is not running
The port use by nifi is already used by another process.
you can change web server port in conf/nifi.properties
I've seen this same issue in several different flavors. The solutions posted however either have nothing to do with my issue or there is simply no solution posted at all.
Apache Nifi 1.5
Java JDK 9.0.4 - Is this version supported by NiFi?
JAVA_HOME=C:\Program Files\Java\jdk-9.0.4
I had an idea that perhaps Apache NiFi doesn't work with JDK version 9.0.4 however I am unable to find any documentation that states what version of JDK Apache NiFi is compatible with and so my assumption is that it doesn't matter.
From what I can see in the nifi-app.log and the nifi-bootstrap.log files have Warnings and Errors all related to 'Permissions'. Another possible cause (guessing) could be the due to the security properties in the nifi.properties file. I have set the nifi.sensitive.propr.key however I see another attribute that I did not set: nifi.sensitive.props.key.protected.
I've placed below the contents of both of the log files as there is nothing in the nifi-user.log file:
nifi-app.log
2018-03-20 14:29:01,599 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2018-03-20 14:29:01,795 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be 'C:\NIFI-1~1.0\.\conf\nifi.properties'
2018-03-20 14:29:01,799 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 144 properties from C:\NIFI-1~1.0\.\conf\nifi.properties
2018-03-20 14:29:01,806 INFO [main] org.apache.nifi.NiFi Loaded 144 properties
2018-03-20 14:29:01,902 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 50794
2018-03-20 14:29:01,920 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2018-03-20 14:29:01,941 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding 89 NAR files with all processors...
2018-03-20 14:29:06,675 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading process took 4734149236 nanoseconds (4 seconds).
2018-03-20 14:29:07,889 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: stackoverflow text removed due to 30000 char limit
2018-03-20 14:29:07,905 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,907 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,910 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,914 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,916 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,917 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,920 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,923 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,924 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,927 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,930 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,933 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,936 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,938 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,945 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,948 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,949 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,950 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,951 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,952 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,953 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,954 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,955 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,956 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,958 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,959 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,960 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,961 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,962 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,963 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,964 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,965 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,966 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,968 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,969 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,971 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,971 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,972 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,973 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,977 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,980 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,982 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,983 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,989 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,990 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,992 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,997 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,998 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,998 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,001 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,002 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,002 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,002 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,004 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,005 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,006 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,008 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,009 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,010 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,011 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,012 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,014 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,016 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,017 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,018 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,019 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,020 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,021 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,021 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,022 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,023 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,024 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,026 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,028 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,029 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,031 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,032 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,033 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,035 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,036 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,037 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,038 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,042 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,043 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,050 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,053 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,061 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,062 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: stackoverflow text removed due to 30000 char limit
2018-03-20 14:29:08,209 INFO [main] org.eclipse.jetty.util.log Logging initialized #7174ms to org.eclipse.jetty.util.log.Slf4jLog
2018-03-20 14:29:08,278 INFO [main] o.a.n.w.s.HostHeaderSanitizationCustomizer Created HostHeaderSanitizationCustomizer for :9090
2018-03-20 14:29:08,278 INFO [main] org.apache.nifi.web.server.JettyServer Added HostHeaderSanitizationCustomizer to HttpConfiguration: HostHeaderSanitizationCustomizer for :9090
2018-03-20 14:29:08,278 INFO [main] org.apache.nifi.web.server.JettyServer Configuring Jetty for HTTP on port: 9090
2018-03-20 14:29:08,410 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-standard-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-jolt-transform-json-ui-1.5.0.war with context path set to /nifi-jolt-transform-json-ui-1.5.0
2018-03-20 14:29:08,410 INFO [main] org.apache.nifi.web.server.JettyServer Loading UI extension [ProcessorConfiguration, /nifi-jolt-transform-json-ui-1.5.0] for org.apache.nifi.processors.standard.JoltTransformJSON:org.apache.nifi:nifi-standard-nar:1.5.0
2018-03-20 14:29:08,413 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-media-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-image-viewer-1.5.0.war with context path set to /nifi-image-viewer-1.5.0
2018-03-20 14:29:08,415 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-update-attribute-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-update-attribute-ui-1.5.0.war with context path set to /nifi-update-attribute-ui-1.5.0
2018-03-20 14:29:08,416 INFO [main] org.apache.nifi.web.server.JettyServer Loading UI extension [ProcessorConfiguration, /nifi-update-attribute-ui-1.5.0] for org.apache.nifi.processors.attributes.UpdateAttribute:org.apache.nifi:nifi-update-attribute-nar:1.5.0
2018-03-20 14:29:08,418 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-standard-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-standard-content-viewer-1.5.0.war with context path set to /nifi-standard-content-viewer-1.5.0
2018-03-20 14:29:08,419 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-ui-1.5.0.war with context path set to /nifi
2018-03-20 14:29:08,420 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-api-1.5.0.war with context path set to /nifi-api
2018-03-20 14:29:08,421 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-content-viewer-1.5.0.war with context path set to /nifi-content-viewer
2018-03-20 14:29:08,423 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-docs-1.5.0.war with context path set to /nifi-docs
2018-03-20 14:29:08,441 INFO [main] org.apache.nifi.web.server.JettyServer Loading documents web app with context path set to /nifi-docs
2018-03-20 14:29:08,447 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-error-1.5.0.war with context path set to /
2018-03-20 14:29:08,471 INFO [main] o.a.nifi.web.server.HostHeaderHandler Created HostHeaderHandler for localhost:9090
2018-03-20 14:29:08,471 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for localhost:9090]
2018-03-20 14:29:08,566 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:581)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:672)
at java.base/java.util.ServiceLoader.access$1000(ServiceLoader.java:390)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1227)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1259)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1294)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1379)
at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:142)
at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:117)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:792)
at org.apache.nifi.NiFi.<init>(NiFi.java:160)
at org.apache.nifi.NiFi.main(NiFi.java:268)
Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3110)
at java.base/java.lang.Class.getConstructor0(Class.java:3315)
at java.base/java.lang.Class.getConstructor(Class.java:2108)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:659)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:656)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:667)
... 10 common frames omitted
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:466)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:563)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
... 18 common frames omitted
2018-03-20 14:29:08,568 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2018-03-20 14:29:08,569 INFO [Thread-0] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
nifi-bootstrap.log
2018-03-20 14:28:59,621 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: []
2018-03-20 14:28:59,625 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED
2018-03-20 14:28:59,625 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED
2018-03-20 14:28:59,625 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED
2018-03-20 14:29:00,686 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2018-03-20 14:29:00,687 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: C:\NIFI-1~1.0
2018-03-20 14:29:00,688 INFO [main] org.apache.nifi.bootstrap.Command Command: C:\Program Files\Java\jdk-9.0.4\bin\java.exe -classpath C:\NIFI-1~1.0\.\conf;C:\NIFI-1~1.0\.\lib\javax.servlet-api-3.1.0.jar;C:\NIFI-1~1.0\.\lib\jcl-over-slf4j-1.7.25.jar;C:\NIFI-1~1.0\.\lib\jetty-schemas-3.1.jar;C:\NIFI-1~1.0\.\lib\jul-to-slf4j-1.7.25.jar;C:\NIFI-1~1.0\.\lib\log4j-over-slf4j-1.7.25.jar;C:\NIFI-1~1.0\.\lib\logback-classic-1.2.3.jar;C:\NIFI-1~1.0\.\lib\logback-core-1.2.3.jar;C:\NIFI-1~1.0\.\lib\nifi-api-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-framework-api-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-nar-utils-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-properties-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-runtime-1.5.0.jar;C:\NIFI-1~1.0\.\lib\slf4j-api-1.7.25.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -XX:+UseG1GC -Dnifi.properties.file.path=C:\NIFI-1~1.0\.\conf\nifi.properties -Dnifi.bootstrap.listen.port=50793 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=C:\NIFI-1~1.0\bin\..\\logs org.apache.nifi.NiFi
2018-03-20 14:29:01,142 WARN [main] org.apache.nifi.bootstrap.Command Failed to set permissions so that only the owner can read pid file C:\NIFI-1~1.0\bin\..\run\nifi.pid; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,146 WARN [main] org.apache.nifi.bootstrap.Command Failed to set permissions so that only the owner can read status file C:\NIFI-1~1.0\bin\..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,148 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 5328
2018-03-20 14:29:01,910 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read pid file C:\NIFI-1~1.0\bin\..\run\nifi.pid; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,916 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read status file C:\NIFI-1~1.0\bin\..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,918 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 50794
2018-03-20 14:29:09,158 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
Two main things I see in these log files that I do not fully understand that can perhaps shine some light on my issue:
app.log: java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
bootstrap log: Permissions should be changed so that only the owner can read this file
Who is the owner? I'm running as an Admin user. Is this referring to the Security Settings set on the file for Windows users?
The issue preventing start-up is the error in nifi-app.log related to the access policy provider.
Specifically the root cause of the exception:
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException
This is likely because of this:
How to resolve java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException in Java 9
The NiFi Admin Guide - System Requirements state that NiFi needs JDK 8 or newer:
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#system-requirements
Technically I believe this should just say JDK 8 since there will have to be some changes to support JDK 9.
I'm trying to export data from teradata to hadoop. but my export query is failing by giving an error "Failed to write data".Please look at the Mapreduce and application logs below:
Log Type: syslog
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 4931
2016-03-08 22:47:07,414 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties
2016-03-08 22:47:07,499 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2016-03-08 22:47:07,499 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2016-03-08 22:47:07,509 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2016-03-08 22:47:07,510 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1457504560070_0004, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier#175b9425)
2016-03-08 22:47:07,556 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: RM_DELEGATION_TOKEN, Service: 39.7.48.2:8032,39.7.48.3:8032, Ident: (owner=hive, renewer=oozie mr token, realUser=oozie, issueDate=1457506410968, maxDate=1458111210968, sequenceNumber=908, masterKeyId=280)
2016-03-08 22:47:07,599 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2016-03-08 22:47:07,848 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data1/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data2/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data3/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data4/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data5/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data6/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data7/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data8/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data9/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data10/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data12/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004
2016-03-08 22:47:08,132 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2016-03-08 22:47:08,623 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]
2016-03-08 22:47:08,840 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: com.teradata.dynaload.hcatalog.mapper.TDInputFormat$TeradataInputSplit#2ece4966
2016-03-08 22:47:08,844 INFO [main] com.teradata.dynaload.hcatalog.mapper.TDInputFormat$TeradataRecordReader: recordreader class com.teradata.dynaload.hcatalog.mapper.TDInputFormat$TeradataRecordReaderinitialize time is: 1457506028844
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 300417020(1201668080)
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 1146
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 841167680
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 1201668096
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 300417020; length = 75104256
2016-03-08 22:47:09,515 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2016-03-08 22:47:09,518 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2016-03-08 22:47:09,518 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
2016-03-08 22:47:09,848 WARN [main] org.apache.hadoop.hive.conf.HiveConf: HiveConf of name hive.metastore.local does not exist
2016-03-08 22:47:09,914 INFO [main] hive.metastore: Trying to connect to metastore with URI thrift://apus2.labs.teradata.com:9083
2016-03-08 22:47:09,951 INFO [main] hive.metastore: Connected to metastore.
2016-03-08 22:47:10,407 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
2016-03-08 22:47:10,452 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2016-03-08 22:47:10,453 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir
2016-03-08 22:47:10,453 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
2016-03-08 22:47:10,457 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
APPLICATION Master LOGS:
Log Type: stderr
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 240
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 0
Log Type: syslog
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 66959
Showing 4096 bytes of 66959 total. Click here for the full log.
ILED
2016-03-08 22:59:19,325 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event JOB_FAILED
2016-03-08 22:59:19,456 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://C423A:8020/user/hive/.staging/job_1457504560070_0004/job_1457504560070_0004_1.jhist to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist_tmp
2016-03-08 22:59:19,550 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist_tmp
2016-03-08 22:59:19,562 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://C423A:8020/user/hive/.staging/job_1457504560070_0004/job_1457504560070_0004_1_conf.xml to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml_tmp
2016-03-08 22:59:19,614 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml_tmp
2016-03-08 22:59:19,645 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004.summary_tmp to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004.summary
2016-03-08 22:59:19,654 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml_tmp to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml
2016-03-08 22:59:19,666 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist_tmp to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist
2016-03-08 22:59:19,666 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2016-03-08 22:59:19,671 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to Task failed task_1457504560070_0004_m_000004
Job failed as tasks failed. failedMaps:1 failedReduces:0
2016-03-08 22:59:19,672 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://apus2.labs.teradata.com:19888/jobhistory/job/job_1457504560070_0004
2016-03-08 22:59:19,680 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.
2016-03-08 22:59:20,682 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:7 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:7 ContRel:0 HostLocal:6 RackLocal:1
2016-03-08 22:59:20,684 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://C423A /user/hive/.staging/job_1457504560070_0004
2016-03-08 22:59:20,711 INFO [Thread-89] org.apache.hadoop.ipc.Server: Stopping server on 46067
2016-03-08 22:59:20,712 INFO [IPC Server listener on 46067] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 46067
2016-03-08 22:59:20,712 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2016-03-08 22:59:20,714 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted.
Plese help me in resolving the issue.
You must be using sqoop to bring data to hadoop. Please paste command you are running. For "Failed to write data" , there can be multiple issues. destination parent directory is not avialble, space is not there at cluster etc.Only command can give explanation.
Previously I have run spark job using command line and spark-submit command.
sudo -su root spark-submit --executor-memory 512m --num-executors 50 --class com.mycompany.project.SparkHdfsToHBase --master yarn parser-1.0-jar-with-dependencies.jar 0 hdfs://company.com:8020/tmp/testfiles/html_files/* 127.0.0.1 --verbose
And everything works like a charm.
Now I am trying to run spark job through oozie workflow. I have created workflow and add shell action with a python script.
Python script looks like:
#!/usr/bin/env python
import subprocess
subprocess.call(["spark-submit", "--class", "com.mycompany.project.SparkHdfsToHBase", "--executor-memory", "512m", "--driver-memory", "512m", "--master", "yarn", "file:///parser-1.0-jar-with-dependencies.jar", "0", "hdfs://company.com:8020/tmp/testfiles/html_files/*", "127.0.0.1", "--verbose"])
I haven't added any other parameters for job, only path to file. When I am submitting job, at job browser my job is stucked on 5%.
I haven't got any useful information from logs:
Syslog:
2015-03-16 11:46:51,152 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-03-16 11:46:51,152 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf; Ignoring.
2015-03-16 11:46:51,154 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class; Ignoring.
2015-03-16 11:46:51,157 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf; Ignoring.
2015-03-16 11:46:51,172 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-03-16 11:46:51,354 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
2015-03-16 11:46:51,354 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN, Service: , Ident: (org.apache.hadoop.yarn.security.AMRMTokenIdentifier#46ea3050)
2015-03-16 11:46:51,393 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: RM_DELEGATION_TOKEN, Service: 172.24.4.231:8032, Ident: (owner=admin, renewer=oozie mr token, realUser=oozie, issueDate=1426499201955, maxDate=1427104001955, sequenceNumber=6, masterKeyId=2)
2015-03-16 11:46:51,631 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert; Ignoring.
2015-03-16 11:46:51,633 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-03-16 11:46:51,637 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf; Ignoring.
2015-03-16 11:46:51,640 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class; Ignoring.
2015-03-16 11:46:51,644 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf; Ignoring.
2015-03-16 11:46:51,661 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-03-16 11:46:52,495 WARN [main] org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-03-16 11:46:52,803 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null
2015-03-16 11:46:52,805 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-03-16 11:46:52,868 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler
2015-03-16 11:46:52,869 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher
2015-03-16 11:46:52,870 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher
2015-03-16 11:46:52,871 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher
2015-03-16 11:46:52,872 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler
2015-03-16 11:46:52,873 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher
2015-03-16 11:46:52,874 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter
2015-03-16 11:46:52,875 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter
2015-03-16 11:46:52,967 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler
2015-03-16 11:46:53,241 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-03-16 11:46:53,303 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-03-16 11:46:53,303 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started
2015-03-16 11:46:53,320 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1426498753001_0001 to jobTokenSecretManager
2015-03-16 11:46:53,447 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1426498753001_0001 because: not enabled;
2015-03-16 11:46:53,473 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1426498753001_0001 = 0. Number of splits = 1
2015-03-16 11:46:53,473 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1426498753001_0001 = 0
2015-03-16 11:46:53,473 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1426498753001_0001Job Transitioned from NEW to INITED
2015-03-16 11:46:53,474 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1426498753001_0001.
2015-03-16 11:46:53,532 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2015-03-16 11:46:53,542 INFO [Socket Reader #1 for port 44690] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 44690
2015-03-16 11:46:53,564 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server
2015-03-16 11:46:53,565 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-03-16 11:46:53,566 INFO [IPC Server listener on 44690] org.apache.hadoop.ipc.Server: IPC Server listener on 44690: starting
2015-03-16 11:46:53,566 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at company.com/172.24.4.231:44690
2015-03-16 11:46:53,641 INFO [main] org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-03-16 11:46:53,646 INFO [main] org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.mapreduce is not defined
2015-03-16 11:46:53,658 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2015-03-16 11:46:53,663 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce
2015-03-16 11:46:53,663 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static
2015-03-16 11:46:53,667 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /mapreduce/*
2015-03-16 11:46:53,667 INFO [main] org.apache.hadoop.http.HttpServer2: adding path spec: /ws/*
2015-03-16 11:46:53,678 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 57328
2015-03-16 11:46:53,678 INFO [main] org.mortbay.log: jetty-6.1.26.cloudera.4
2015-03-16 11:46:53,706 INFO [main] org.mortbay.log: Extract jar:file:/opt/cloudera/parcels/CDH-5.3.1-1.cdh5.3.1.p0.5/jars/hadoop-yarn-common-2.5.0-cdh5.3.1.jar!/webapps/mapreduce to /tmp/Jetty_0_0_0_0_57328_mapreduce____37uqf6/webapp
2015-03-16 11:46:54,346 INFO [main] org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup#0.0.0.0:57328
2015-03-16 11:46:54,346 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app /mapreduce started at 57328
2015-03-16 11:46:55,005 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules
2015-03-16 11:46:55,012 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2015-03-16 11:46:55,012 INFO [Socket Reader #1 for port 60762] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 60762
2015-03-16 11:46:55,021 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2015-03-16 11:46:55,023 INFO [IPC Server listener on 60762] org.apache.hadoop.ipc.Server: IPC Server listener on 60762: starting
2015-03-16 11:46:55,111 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true
2015-03-16 11:46:55,111 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3
2015-03-16 11:46:55,111 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33
2015-03-16 11:46:55,369 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert; Ignoring.
2015-03-16 11:46:55,369 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-03-16 11:46:55,369 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf; Ignoring.
2015-03-16 11:46:55,370 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class; Ignoring.
2015-03-16 11:46:55,375 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf; Ignoring.
2015-03-16 11:46:55,383 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-03-16 11:46:55,388 INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at company.com/172.24.4.231:8030
2015-03-16 11:46:55,574 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: maxContainerCapability: <memory:8192, vCores:2>
2015-03-16 11:46:55,574 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: queue: root.admin
2015-03-16 11:46:55,582 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500
2015-03-16 11:46:55,584 INFO [main] org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
2015-03-16 11:46:55,599 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1426498753001_0001Job Transitioned from INITED to SETUP
2015-03-16 11:46:55,611 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP
2015-03-16 11:46:55,633 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1426498753001_0001Job Transitioned from SETUP to RUNNING
2015-03-16 11:46:55,714 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1426498753001_0001_m_000000 Task Transitioned from NEW to SCHEDULED
2015-03-16 11:46:55,723 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1426498753001_0001_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED
2015-03-16 11:46:55,738 INFO [Thread-50] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:<memory:1024, vCores:1>
2015-03-16 11:46:55,797 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1426498753001_0001, File: hdfs://company.com:8020/user/admin/.staging/job_1426498753001_0001/job_1426498753001_0001_1.jhist
2015-03-16 11:46:56,584 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0
2015-03-16 11:46:56,658 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1426498753001_0001: ask=1 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:7168, vCores:0> knownNMs=1
Stderr:
Mar 16, 2015 11:47:04 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Mar 16, 2015 11:47:04 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Mar 16, 2015 11:47:04 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Mar 16, 2015 11:47:04 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Mar 16, 2015 11:47:04 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Mar 16, 2015 11:47:05 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Mar 16, 2015 11:47:05 AM com.google.inject.servlet.InternalServletModule$BackwardsCompatibleServletContextProvider get
WARNING: You are attempting to use a deprecated API (specifically, attempting to #Inject ServletContext inside an eagerly created singleton. While we allow this for backwards compatibility, be warned that this MAY have unexpected behavior if you have more than one injector (with ServletModule) running in the same JVM. Please consult the Guice documentation at http://code.google.com/p/google-guice/wiki/Servlets for more information.
Mar 16, 2015 11:47:05 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
I'll be happy to get any help. Thanks.