I've seen this same issue in several different flavors. The solutions posted however either have nothing to do with my issue or there is simply no solution posted at all.
Apache Nifi 1.5
Java JDK 9.0.4 - Is this version supported by NiFi?
JAVA_HOME=C:\Program Files\Java\jdk-9.0.4
I had an idea that perhaps Apache NiFi doesn't work with JDK version 9.0.4 however I am unable to find any documentation that states what version of JDK Apache NiFi is compatible with and so my assumption is that it doesn't matter.
From what I can see in the nifi-app.log and the nifi-bootstrap.log files have Warnings and Errors all related to 'Permissions'. Another possible cause (guessing) could be the due to the security properties in the nifi.properties file. I have set the nifi.sensitive.propr.key however I see another attribute that I did not set: nifi.sensitive.props.key.protected.
I've placed below the contents of both of the log files as there is nothing in the nifi-user.log file:
nifi-app.log
2018-03-20 14:29:01,599 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2018-03-20 14:29:01,795 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be 'C:\NIFI-1~1.0\.\conf\nifi.properties'
2018-03-20 14:29:01,799 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 144 properties from C:\NIFI-1~1.0\.\conf\nifi.properties
2018-03-20 14:29:01,806 INFO [main] org.apache.nifi.NiFi Loaded 144 properties
2018-03-20 14:29:01,902 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 50794
2018-03-20 14:29:01,920 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2018-03-20 14:29:01,941 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding 89 NAR files with all processors...
2018-03-20 14:29:06,675 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading process took 4734149236 nanoseconds (4 seconds).
2018-03-20 14:29:07,889 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: stackoverflow text removed due to 30000 char limit
2018-03-20 14:29:07,905 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,907 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,910 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,914 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,916 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,917 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,920 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,923 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,924 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,927 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,930 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,933 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,936 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,938 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,945 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,948 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,949 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,950 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,951 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,952 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,953 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,954 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,955 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,956 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,958 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,959 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,960 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,961 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,962 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,963 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,964 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,965 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,966 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,968 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,969 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,971 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,971 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,972 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,973 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,977 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,980 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,982 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,983 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,989 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,990 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,992 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,997 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,998 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:07,998 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,001 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,002 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,002 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,002 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,004 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,005 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,006 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,008 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,009 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,010 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,011 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,012 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,014 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,016 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,017 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,018 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,019 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,020 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,021 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,021 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,022 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,023 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,024 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,026 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,028 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,029 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,031 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,032 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,033 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,035 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,036 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,037 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,038 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,042 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,043 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,050 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,053 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,061 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: ""
2018-03-20 14:29:08,062 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: stackoverflow text removed due to 30000 char limit
2018-03-20 14:29:08,209 INFO [main] org.eclipse.jetty.util.log Logging initialized #7174ms to org.eclipse.jetty.util.log.Slf4jLog
2018-03-20 14:29:08,278 INFO [main] o.a.n.w.s.HostHeaderSanitizationCustomizer Created HostHeaderSanitizationCustomizer for :9090
2018-03-20 14:29:08,278 INFO [main] org.apache.nifi.web.server.JettyServer Added HostHeaderSanitizationCustomizer to HttpConfiguration: HostHeaderSanitizationCustomizer for :9090
2018-03-20 14:29:08,278 INFO [main] org.apache.nifi.web.server.JettyServer Configuring Jetty for HTTP on port: 9090
2018-03-20 14:29:08,410 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-standard-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-jolt-transform-json-ui-1.5.0.war with context path set to /nifi-jolt-transform-json-ui-1.5.0
2018-03-20 14:29:08,410 INFO [main] org.apache.nifi.web.server.JettyServer Loading UI extension [ProcessorConfiguration, /nifi-jolt-transform-json-ui-1.5.0] for org.apache.nifi.processors.standard.JoltTransformJSON:org.apache.nifi:nifi-standard-nar:1.5.0
2018-03-20 14:29:08,413 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-media-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-image-viewer-1.5.0.war with context path set to /nifi-image-viewer-1.5.0
2018-03-20 14:29:08,415 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-update-attribute-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-update-attribute-ui-1.5.0.war with context path set to /nifi-update-attribute-ui-1.5.0
2018-03-20 14:29:08,416 INFO [main] org.apache.nifi.web.server.JettyServer Loading UI extension [ProcessorConfiguration, /nifi-update-attribute-ui-1.5.0] for org.apache.nifi.processors.attributes.UpdateAttribute:org.apache.nifi:nifi-update-attribute-nar:1.5.0
2018-03-20 14:29:08,418 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\extensions\nifi-standard-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-standard-content-viewer-1.5.0.war with context path set to /nifi-standard-content-viewer-1.5.0
2018-03-20 14:29:08,419 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-ui-1.5.0.war with context path set to /nifi
2018-03-20 14:29:08,420 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-api-1.5.0.war with context path set to /nifi-api
2018-03-20 14:29:08,421 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-content-viewer-1.5.0.war with context path set to /nifi-content-viewer
2018-03-20 14:29:08,423 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-docs-1.5.0.war with context path set to /nifi-docs
2018-03-20 14:29:08,441 INFO [main] org.apache.nifi.web.server.JettyServer Loading documents web app with context path set to /nifi-docs
2018-03-20 14:29:08,447 INFO [main] org.apache.nifi.web.server.JettyServer Loading WAR: C:\NIFI-1~1.0\.\work\nar\framework\nifi-framework-nar-1.5.0.nar-unpacked\META-INF\bundled-dependencies\nifi-web-error-1.5.0.war with context path set to /
2018-03-20 14:29:08,471 INFO [main] o.a.nifi.web.server.HostHeaderHandler Created HostHeaderHandler for localhost:9090
2018-03-20 14:29:08,471 INFO [main] org.apache.nifi.web.server.JettyServer Created HostHeaderHandler [HostHeaderHandler for localhost:9090]
2018-03-20 14:29:08,566 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:581)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:672)
at java.base/java.util.ServiceLoader.access$1000(ServiceLoader.java:390)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1227)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1259)
at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1294)
at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1379)
at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:142)
at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:117)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:792)
at org.apache.nifi.NiFi.<init>(NiFi.java:160)
at org.apache.nifi.NiFi.main(NiFi.java:268)
Caused by: java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3110)
at java.base/java.lang.Class.getConstructor0(Class.java:3315)
at java.base/java.lang.Class.getConstructor(Class.java:2108)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:659)
at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:656)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:667)
... 10 common frames omitted
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:466)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:563)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:496)
... 18 common frames omitted
2018-03-20 14:29:08,568 INFO [Thread-0] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2018-03-20 14:29:08,569 INFO [Thread-0] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
nifi-bootstrap.log
2018-03-20 14:28:59,621 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: []
2018-03-20 14:28:59,625 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED
2018-03-20 14:28:59,625 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED
2018-03-20 14:28:59,625 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED
2018-03-20 14:29:00,686 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2018-03-20 14:29:00,687 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: C:\NIFI-1~1.0
2018-03-20 14:29:00,688 INFO [main] org.apache.nifi.bootstrap.Command Command: C:\Program Files\Java\jdk-9.0.4\bin\java.exe -classpath C:\NIFI-1~1.0\.\conf;C:\NIFI-1~1.0\.\lib\javax.servlet-api-3.1.0.jar;C:\NIFI-1~1.0\.\lib\jcl-over-slf4j-1.7.25.jar;C:\NIFI-1~1.0\.\lib\jetty-schemas-3.1.jar;C:\NIFI-1~1.0\.\lib\jul-to-slf4j-1.7.25.jar;C:\NIFI-1~1.0\.\lib\log4j-over-slf4j-1.7.25.jar;C:\NIFI-1~1.0\.\lib\logback-classic-1.2.3.jar;C:\NIFI-1~1.0\.\lib\logback-core-1.2.3.jar;C:\NIFI-1~1.0\.\lib\nifi-api-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-framework-api-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-nar-utils-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-properties-1.5.0.jar;C:\NIFI-1~1.0\.\lib\nifi-runtime-1.5.0.jar;C:\NIFI-1~1.0\.\lib\slf4j-api-1.7.25.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -XX:+UseG1GC -Dnifi.properties.file.path=C:\NIFI-1~1.0\.\conf\nifi.properties -Dnifi.bootstrap.listen.port=50793 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=C:\NIFI-1~1.0\bin\..\\logs org.apache.nifi.NiFi
2018-03-20 14:29:01,142 WARN [main] org.apache.nifi.bootstrap.Command Failed to set permissions so that only the owner can read pid file C:\NIFI-1~1.0\bin\..\run\nifi.pid; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,146 WARN [main] org.apache.nifi.bootstrap.Command Failed to set permissions so that only the owner can read status file C:\NIFI-1~1.0\bin\..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,148 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 5328
2018-03-20 14:29:01,910 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read pid file C:\NIFI-1~1.0\bin\..\run\nifi.pid; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,916 WARN [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Failed to set permissions so that only the owner can read status file C:\NIFI-1~1.0\bin\..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2018-03-20 14:29:01,918 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 50794
2018-03-20 14:29:09,158 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
Two main things I see in these log files that I do not fully understand that can perhaps shine some light on my issue:
app.log: java.util.ServiceConfigurationError: org.apache.nifi.authorization.AccessPolicyProvider: org.apache.nifi.authorization.FileAccessPolicyProvider Unable to get public no-arg constructor
bootstrap log: Permissions should be changed so that only the owner can read this file
Who is the owner? I'm running as an Admin user. Is this referring to the Security Settings set on the file for Windows users?
The issue preventing start-up is the error in nifi-app.log related to the access policy provider.
Specifically the root cause of the exception:
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.JAXBException
This is likely because of this:
How to resolve java.lang.NoClassDefFoundError: javax/xml/bind/JAXBException in Java 9
The NiFi Admin Guide - System Requirements state that NiFi needs JDK 8 or newer:
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#system-requirements
Technically I believe this should just say JDK 8 since there will have to be some changes to support JDK 9.
Related
When I tried to connect to Nifi UI using http://localhost:8080/nifi, i am getting below error
org.apache.nifi.web.server.JettyServer Failed to start web server... shutting down.
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:331)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:299)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:398)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:935)
at org.apache.nifi.NiFi.<init>(NiFi.java:158)
at org.apache.nifi.NiFi.<init>(NiFi.java:72)
at org.apache.nifi.NiFi.main(NiFi.java:297)
2020-02-27 11:51:11,834 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2020-02-27 11:51:11,836 INFO [Thread-1] o.eclipse.jetty.server.AbstractConnector Stopped ServerConnector#355ee205{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
2020-02-27 11:51:11,837 INFO [Thread-1] org.eclipse.jetty.server.session node 0 Stopped scavenging
Can anyone suggest what is the cause of this issue?
Nifi version- 1.9.2,installed on windows machine
Here is the nifi status logs,
12:33:16.886 [main] DEBUG org.apache.nifi.bootstrap.NotificationServiceManager - Found 0 service elements
12:33:16.896 [main] INFO org.apache.nifi.bootstrap.NotificationServiceManager - Successfully loaded the following 0 services: []
12:33:16.897 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_STARTED
12:33:16.897 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_STOPPED
12:33:16.898 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_DIED
12:33:16.899 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File:
12:33:16.900 [main] DEBUG org.apache.nifi.bootstrap.Command - Properties: {pid=9724}
Failed to determine if Process 9724 is running; assuming that it is not
12:33:16.902 [main] INFO org.apache.nifi.bootstrap.Command - Apache NiFi is not running
The port use by nifi is already used by another process.
you can change web server port in conf/nifi.properties
I'll comment on the problem i'm having.
In the company where I work, they decided a few years ago to implement nifi.
The problem with this is that they implemented the complete HDF solution, something really unnecessary to use only nifi.
With luck the Ambari interface is used to restart the service
I am currently in the process of migrating this nifi that consists of 3 clustered nodes (something also unnecessary since only small tasks are performed and resources are wasted) to a dockerized nifi standalone.
This process was done previously with a nifi with a smaller flow successfully.
The procedure that was carried out was simply to export a template of the flow and to import it, adding in the nifi.properties the correct keys to be able to use the controller services.
I find myself trying to perform this same procedure without success.
The flow is imported correctly but I can not visualize the controllers services that I have loaded by adding the correct key to the nifi.properties.
Try extracting the key from the controller services for load manually that appears as "sensitive value" but I could not do it.
As 2nd option copy the flow.xml.gz of the server to be migrated, edit some fields with a text editor (such as file paths, drivers, etc) and apparently I'm better with this;
I had several errors, among which I had to add the drivers (jdbc, hive, etc). I'm correcting small problems little by little.
Is there really any correct way to migrate this?
Because for more than correctly entering the password in the nifi.properties file, I can not see the controller services? Does HDF have any other configuration files?
As an extra data, the version of dockerized nifi used is 1.2.0, the version of nifi that currently has HDF is 1.1.0. Apparently there are no major changes that could affect.
Thanks for the help!
The sensitive properties in the flow.xml.gz file are encrypted with a key derived from nifi.sensitive.props.key entry in nifi.properties. If this value is itself encrypted (i.e. it is of the form abcdef...||abcdef...), you will also need to copy the nifi.bootstrap.sensitive.key value from bootstrap.conf. With those two values (and using the same algorithm for KDF and encryption as specified in nifi.sensitive.props.algorithm in nifi.properties), your new NiFi instance should be able to decrypt the sensitive processor properties that are embedded in an encrypted form in flow.xml.gz.
Thanks for the help
I did not know this about the master key in the bootstrap.conf file
I have done the following:
- Export from the original nifi the flow, and import it into the new nifi.
- Edit indicating the keys in the nifi.properties and bootstrap.conf
- The service does not start, the logs announce the following
2019-06-06 17:28:21,807 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: []
2019-06-06 17:28:21,809 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED
2019-06-06 17:28:21,809 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED
2019-06-06 17:28:21,809 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED
2019-06-06 17:28:21,829 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2019-06-06 17:28:21,829 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /opt/nifi/nifi-1.2.0
2019-06-06 17:28:21,829 INFO [main] org.apache.nifi.bootstrap.Command Command: /docker-java-home/bin/java -classpath /opt/nifi/nifi-1.2.0/./conf:/opt/nifi/nifi-1.2.0/./lib/slf4j-api-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/jul-to-slf4j-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/logback-classic-1.2.3.jar:/opt/nifi/nifi-1.2.0/./lib/jcl-over-slf4j-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-api-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/javax.servlet-api-3.1.0.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-framework-api-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/log4j-over-slf4j-1.7.25.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-properties-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/jetty-schemas-3.1.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-runtime-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/nifi-nar-utils-1.2.0.jar:/opt/nifi/nifi-1.2.0/./lib/logback-core-1.2.3.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/opt/nifi/nifi-1.2.0/./conf/nifi.properties -Dnifi.bootstrap.listen.port=40303 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/opt/nifi/nifi-1.2.0/logs org.apache.nifi.NiFi -K /opt/nifi/nifi-1.2.0/./conf/sensitive.key
2019-06-06 17:28:21,912 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 35
2019-06-06 17:28:22,814 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 35155
2019-06-06 17:28:41,948 ERROR [NiFi logging handler] org.apache.nifi.StdErr Failed to start web server: org.jasypt.exceptions.EncryptionOperationNotPossibleException
2019-06-06 17:28:41,949 ERROR [NiFi logging handler] org.apache.nifi.StdErr Shutting down...
2019-06-06 17:28:42,922 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never started. Will not restart NiFi
After this it seems the service keeps restarting so replace the nifi-framework-nar file with version 1.1.0 so that it does not have the correct jasypt library. (Remember with i having migrate nifi 1.1 to 1.2)
Nifi keeps restarting announcing the following error.
2019-06-06 18:25:18,143 INFO [main] org.apache.nifi.NiFi Launching NiFi...
2019-06-06 18:25:18,208 INFO [main] org.apache.nifi.NiFi Now overwriting file in /opt/nifi/nifi-1.2.0/./conf/sensitive.key
2019-06-06 18:25:18,209 INFO [main] org.apache.nifi.NiFi Overwriting key material with pad: c957a7de1ce10bf7a2c1b8cc8907df203e13bfb497e75b81e6fcd8cfbcae449d
2019-06-06 18:25:18,209 INFO [main] org.apache.nifi.NiFi Removing/unlinking file: /opt/nifi/nifi-1.2.0/./conf/sensitive.key
2019-06-06 18:25:18,209 INFO [main] org.apache.nifi.NiFi Read property protection key from key file provided by bootstrap process
2019-06-06 18:25:18,405 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Determined default nifi.properties path to be '/opt/nifi/nifi-1.2.0/./conf/nifi.properties'
2019-06-06 18:25:18,411 INFO [main] o.a.nifi.properties.NiFiPropertiesLoader Loaded 124 properties from /opt/nifi/nifi-1.2.0/./conf/nifi.properties
2019-06-06 18:25:18,597 INFO [main] o.a.n.properties.ProtectedNiFiProperties There are 1 protected properties of 5 sensitive properties (100%)
2019-06-06 18:25:18,604 INFO [main] o.a.n.p.AESSensitivePropertyProvider AES Sensitive Property Provider decrypted a sensitive value successfully
2019-06-06 18:25:18,605 INFO [main] org.apache.nifi.NiFi Loaded 123 properties
2019-06-06 18:25:18,614 INFO [main] org.apache.nifi.BootstrapListener Started Bootstrap Listener, Listening for incoming requests on port 34369
2019-06-06 18:25:18,696 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap
2019-06-06 18:25:18,718 INFO [main] org.apache.nifi.nar.NarUnpacker Expanding 71 NAR files with all processors...
2019-06-06 18:25:22,910 INFO [main] org.apache.nifi.nar.NarUnpacker NAR loading process took 4192250200 nanoseconds.
2019-06-06 18:25:23,177 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-jetty-bundle-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jetty-bundle-1.2.0.nar-unpacked]
2019-06-06 18:25:23,186 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/framework/nifi-framework-nar-1.1.0.2.1.0.0-165.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/framework/nifi-framework-nar-1.1.0.2.1.0.0-165.nar-unpacked]
2019-06-06 18:25:23,188 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ldap-iaa-providers-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ldap-iaa-providers-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,193 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-gcp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-gcp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,194 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kerberos-iaa-providers-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kerberos-iaa-providers-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,195 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-standard-services-api-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-services-api-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,196 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hl7-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hl7-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,197 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-datadog-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-datadog-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,198 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-solr-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-solr-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,199 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-websocket-services-api-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-websocket-services-api-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,201 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ccda-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ccda-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,202 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-amqp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-amqp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,207 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-aws-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-aws-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,207 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-dbcp-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-dbcp-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,209 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-email-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-email-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,210 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-cassandra-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-cassandra-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,211 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-registry-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-registry-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,212 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kafka-0-10-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kafka-0-10-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,212 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kafka-0-9-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kafka-0-9-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,215 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-media-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-media-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,216 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-websocket-processors-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-websocket-processors-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,217 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-avro-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-avro-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,218 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-scripting-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-scripting-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,218 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ssl-context-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ssl-context-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,218 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-couchbase-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-couchbase-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,219 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-social-media-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-social-media-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,220 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kafka-0-8-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kafka-0-8-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,220 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-stateful-analysis-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-stateful-analysis-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,221 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-site-to-site-reporting-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-site-to-site-reporting-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,222 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-poi-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-poi-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,223 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-tcp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-tcp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,223 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-mqtt-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-mqtt-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,224 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-enrich-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-enrich-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,226 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hadoop-libraries-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hadoop-libraries-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,227 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-html-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-html-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,227 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-evtx-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-evtx-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,229 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-elasticsearch-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-elasticsearch-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,229 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-snmp-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-snmp-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,230 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-websocket-services-jetty-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-websocket-services-jetty-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,232 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-flume-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-flume-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,232 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-update-attribute-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-update-attribute-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,233 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-cybersecurity-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-cybersecurity-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,233 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-spring-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-spring-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,234 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-azure-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-azure-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,235 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-kite-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-kite-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,236 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-distributed-cache-services-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-distributed-cache-services-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,236 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ignite-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ignite-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,237 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-windows-event-log-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-windows-event-log-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,237 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-lumberjack-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-lumberjack-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,237 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-beats-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-beats-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,238 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-http-context-map-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-http-context-map-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,240 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-standard-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,241 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-elasticsearch-5-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-elasticsearch-5-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,241 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-riemann-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-riemann-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,242 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-slack-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-slack-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,242 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hwx-schema-registry-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hwx-schema-registry-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,243 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hive-services-api-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hive-services-api-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,243 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-provenance-repository-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-provenance-repository-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,243 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-mongodb-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-mongodb-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,244 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-splunk-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-splunk-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,244 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-language-translation-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-language-translation-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,245 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-record-serialization-services-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-record-serialization-services-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,246 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-ambari-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-ambari-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,246 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-jms-cf-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jms-cf-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,247 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-cdc-mysql-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-cdc-mysql-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,247 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hbase-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hbase-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,249 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hbase_1_1_2-client-service-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hbase_1_1_2-client-service-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,250 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hadoop-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hadoop-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,251 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-parquet-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-parquet-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,251 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-jms-processors-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-jms-processors-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,254 INFO [main] org.apache.nifi.nar.NarClassLoaders Loaded NAR file: /opt/nifi/nifi-1.2.0/./work/nar/extensions/nifi-hive-nar-1.2.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-hive-nar-1.2.0.nar-unpacked]
2019-06-06 18:25:23,323 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.lang.NoSuchMethodException: org.apache.nifi.web.server.JettyServer.(org.apache.nifi.util.NiFiProperties, java.util.Set)
2019-06-06 18:25:23,324 INFO [Thread-1] org.apache.nifi.NiFi Initiating shutdown of Jetty web server...
2019-06-06 18:25:23,325 INFO [Thread-1] org.apache.nifi.NiFi Jetty web server shutdown completed (nicely or otherwise).
Delete the folder of the flowfiles but it still does not start.
I did the test of using the folder lib of my nifi 1.1 and the error is the following one: Error: Could not find or load main class org.apache.nifi.bootstrap.RunNiFi
Where are the files specified to be used in nifi?
Apparently for more than amount the flow in the right way, the operation of the processors continues to depend on the files nar with the libraries belonging to its version.
Thanks!
I try to use Hadoop 3.0 GA release with gpu, but when I executed the below shell command, there is an error and not working with gpu. please check the below and just let you know the shell command. I guess that there are misconfigurations from me.
2018-01-09 15:04:49,256 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:main(355)) - Initializing ApplicationMaster
2018-01-09 15:04:49,391 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:init(514)) - Application master for app, appId=1, clustertimestamp=1515477741976, attemptId=1
2018-01-09 15:04:49,418 WARN [main] distributedshell.ApplicationMaster (ApplicationMaster.java:init(626)) - Timeline service is not enabled
2018-01-09 15:04:49,418 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(649)) - Starting ApplicationMaster
2018-01-09 15:04:49,542 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(60)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2018-01-09 15:04:49,623 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(659)) - Executing with tokens:
2018-01-09 15:04:49,744 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(662)) - Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1515477741976 } attemptId: 1 } keyId: 1619387150)
2018-01-09 15:04:49,801 INFO [main] client.RMProxy (RMProxy.java:newProxyInstance(133)) - Connecting to ResourceManager at /0.0.0.0:8030
2018-01-09 15:04:49,886 INFO [main] impl.NMClientAsyncImpl (NMClientAsyncImpl.java:serviceInit(138)) - Upper bound of the thread pool size is 500
2018-01-09 15:04:49,889 WARN [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(786)) - Timeline service is not enabled
2018-01-09 15:04:50,170 INFO [main] conf.Configuration (Configuration.java:getConfResourceAsInputStream(2656)) - resource-types.xml not found
2018-01-09 15:04:50,170 INFO [main] resource.ResourceUtils (ResourceUtils.java:addResourcesFileToConf(395)) - Unable to find 'resource-types.xml'.
2018-01-09 15:04:50,183 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,185 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,185 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,185 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,187 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,187 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,188 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,188 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,188 WARN [main] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:50,188 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(717)) - Max mem capability of resources in this cluster 8192
2018-01-09 15:04:50,188 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(720)) - Max vcores capability of resources in this cluster 4
2018-01-09 15:04:50,189 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:run(739)) - appattempt_1515477741976_0001_000001 received 0 previous attempts' running containers on AM registration.
2018-01-09 15:04:50,202 INFO [main] distributedshell.ApplicationMaster (ApplicationMaster.java:setupContainerAskForRM(1311)) - Requested container ask: Capability[<memory:-1, vCores:-1>]Priority[0]AllocationRequestId[0]ExecutionTypeRequest[{Execution Type: GUARANTEED, Enforce Execution Type: false}]Resource Profile[gpu-1]
2018-01-09 15:04:50,246 WARN [AMRM Heartbeater thread] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:51,255 WARN [AMRM Heartbeater thread] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:52,273 WARN [AMRM Heartbeater thread] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
2018-01-09 15:04:52,278 INFO [AMRM Callback Handler Thread] distributedshell.ApplicationMaster (ApplicationMaster.java:onContainersAllocated(957)) - Got response from RM for container ask, allocatedCnt=1
2018-01-09 15:04:52,278 WARN [AMRM Callback Handler Thread] pb.ResourcePBImpl (ResourcePBImpl.java:initResources(142)) - Got unknown resource type: yarn.io/gpu; skipping
And the shell command that I executed with respect to YARN-7223 ticket is followed by,
yarn jar <path/to/hadoop-yarn-applications-distributedshell.jar> \ -jar <path/to/hadoop-yarn-applications-distributedshell.jar> \ -shell_command /usr/local/nvidia/bin/nvidia-smi -container_resource_profile gpu-1
Thanks in advance.
I have a problem i am trying to execute pig command in local mode I am using pig -x local and trying to execute a dump statment but its giving me error please have a look and explain to me, another problem is when I am running the same command dump b; in mapreduce mode the command is running fine and its showing me the results i wonder why its not running in local mode please have a look:-
[dead#master ~]$ pig -x local
17/04/20 16:00:53 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
17/04/20 16:00:53 INFO pig.ExecTypeProvider: Picked LOCAL as the ExecType
2017-04-20 16:00:53,772 [main] INFO org.apache.pig.Main - Apache Pig version 0.16.0 (r1746530) compiled Jun 01 2016, 23:10:49
2017-04-20 16:00:53,772 [main] INFO org.apache.pig.Main - Logging error messages to: /opt/hadoop/pig_1492684253771.log
2017-04-20 16:00:53,826 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /opt/hadoop/.pigbootup not found
2017-04-20 16:00:54,150 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:///
2017-04-20 16:00:54,322 [main] INFO org.apache.pig.PigServer - Pig Script ID for the session: PIG-default-f6e00222-79c4-4e12-a2f1-c9404e17c69c
2017-04-20 16:00:54,322 [main] WARN org.apache.pig.PigServer - ATS is disabled since yarn.timeline-service.enabled set to false
grunt> data = load '/books.csv' using PigStorage(',') as (booknum:int, author:chararray, title:chararray, published:int);
grunt> b = foreach data generate author,title;
grunt> dump b;
2017-04-20 16:01:06,219 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2017-04-20 16:01:06,424 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2017-04-20 16:01:06,493 [main] INFO org.apache.pig.newplan.logical.rules.ColumnPruneVisitor - Columns pruned for data: $0, $3
2017-04-20 16:01:06,600 [main] INFO org.apache.pig.impl.util.SpillableMemoryManager - Selected heap (Tenured Gen) of size 699072512 to monitor. collectionUsageThreshold = 489350752, usageThreshold = 489350752
2017-04-20 16:01:06,758 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2017-04-20 16:01:06,821 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2017-04-20 16:01:06,821 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2017-04-20 16:01:06,945 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Initializing JVM Metrics with processName=JobTracker, sessionId=
2017-04-20 16:01:07,013 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRScriptState - Pig script settings are added to the job
2017-04-20 16:01:07,049 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2017-04-20 16:01:07,090 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2017-04-20 16:01:07,141 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2017-04-20 16:01:07,141 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche
2017-04-20 16:01:07,141 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1492684267141-0
2017-04-20 16:01:07,242 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
2017-04-20 16:01:07,315 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2017-04-20 16:01:07,763 [JobControl] WARN org.apache.hadoop.mapreduce.JobResourceUploader - No job jar file set. User classes may not be found. See Job or Job#setJar(String).
2017-04-20 16:01:07,813 [JobControl] INFO org.apache.pig.builtin.PigStorage - Using PigTextInputFormat
2017-04-20 16:01:07,860 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Cleaning up the staging area file:/tmp/hadoop-dead/mapred/staging/dead1668871356/.staging/job_local1668871356_0001
2017-04-20 16:01:07,893 [JobControl] INFO org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob - PigLatin:DefaultJobName got an error while submitting
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: file:/books.csv
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:194)
at java.lang.Thread.run(Thread.java:745)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/books.csv
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
... 18 more
2017-04-20 16:01:07,898 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1668871356_0001
2017-04-20 16:01:07,898 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases b,data
2017-04-20 16:01:07,898 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: data[1,7],b[-1,-1] C: R:
2017-04-20 16:01:07,972 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
2017-04-20 16:01:08,113 [main] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
2017-04-20 16:01:08,114 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local1668871356_0001 has failed! Stop running all dependent jobs
2017-04-20 16:01:08,114 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2017-04-20 16:01:08,142 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2017-04-20 16:01:08,142 [main] ERROR org.apache.pig.tools.pigstats.PigStats - ERROR 0: java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
2017-04-20 16:01:08,142 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed!
2017-04-20 16:01:08,143 [main] INFO org.apache.pig.tools.pigstats.mapreduce.SimplePigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
2.7.2 0.16.0 dead 2017-04-20 16:01:07 2017-04-20 16:01:08 UNKNOWN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_local1668871356_0001 b,data MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: file:/books.csv
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:194)
at java.lang.Thread.run(Thread.java:745)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/books.csv
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
... 18 more
file:/tmp/temp-1274993195/tmp-1105974594,
Input(s):
Failed to read data from "/books.csv"
Output(s):
Failed to produce result in "file:/tmp/temp-1274993195/tmp-1105974594"
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local1668871356_0001
2017-04-20 16:01:08,144 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2017-04-20 16:01:08,147 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias b. Backend error : java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
Details at logfile: /opt/hadoop/pig_1492684253771.log
I'm trying to export data from teradata to hadoop. but my export query is failing by giving an error "Failed to write data".Please look at the Mapreduce and application logs below:
Log Type: syslog
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 4931
2016-03-08 22:47:07,414 WARN [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-maptask.properties,hadoop-metrics2.properties
2016-03-08 22:47:07,499 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2016-03-08 22:47:07,499 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2016-03-08 22:47:07,509 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2016-03-08 22:47:07,510 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1457504560070_0004, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier#175b9425)
2016-03-08 22:47:07,556 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: RM_DELEGATION_TOKEN, Service: 39.7.48.2:8032,39.7.48.3:8032, Ident: (owner=hive, renewer=oozie mr token, realUser=oozie, issueDate=1457506410968, maxDate=1458111210968, sequenceNumber=908, masterKeyId=280)
2016-03-08 22:47:07,599 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2016-03-08 22:47:07,848 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /data1/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data2/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data3/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data4/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data5/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data6/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data7/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data8/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data9/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data10/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004,/data12/hadoop/yarn/local/usercache/hive/appcache/application_1457504560070_0004
2016-03-08 22:47:08,132 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2016-03-08 22:47:08,623 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]
2016-03-08 22:47:08,840 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: com.teradata.dynaload.hcatalog.mapper.TDInputFormat$TeradataInputSplit#2ece4966
2016-03-08 22:47:08,844 INFO [main] com.teradata.dynaload.hcatalog.mapper.TDInputFormat$TeradataRecordReader: recordreader class com.teradata.dynaload.hcatalog.mapper.TDInputFormat$TeradataRecordReaderinitialize time is: 1457506028844
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 300417020(1201668080)
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 1146
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 841167680
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 1201668096
2016-03-08 22:47:09,512 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 300417020; length = 75104256
2016-03-08 22:47:09,515 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2016-03-08 22:47:09,518 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2016-03-08 22:47:09,518 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
2016-03-08 22:47:09,848 WARN [main] org.apache.hadoop.hive.conf.HiveConf: HiveConf of name hive.metastore.local does not exist
2016-03-08 22:47:09,914 INFO [main] hive.metastore: Trying to connect to metastore with URI thrift://apus2.labs.teradata.com:9083
2016-03-08 22:47:09,951 INFO [main] hive.metastore: Connected to metastore.
2016-03-08 22:47:10,407 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
2016-03-08 22:47:10,452 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1
2016-03-08 22:47:10,453 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.work.output.dir is deprecated. Instead, use mapreduce.task.output.dir
2016-03-08 22:47:10,453 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
2016-03-08 22:47:10,457 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
APPLICATION Master LOGS:
Log Type: stderr
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 240
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 0
Log Type: syslog
Log Upload Time: Tue Mar 08 22:59:27 -0800 2016
Log Length: 66959
Showing 4096 bytes of 66959 total. Click here for the full log.
ILED
2016-03-08 22:59:19,325 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: In stop, writing event JOB_FAILED
2016-03-08 22:59:19,456 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://C423A:8020/user/hive/.staging/job_1457504560070_0004/job_1457504560070_0004_1.jhist to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist_tmp
2016-03-08 22:59:19,550 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist_tmp
2016-03-08 22:59:19,562 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://C423A:8020/user/hive/.staging/job_1457504560070_0004/job_1457504560070_0004_1_conf.xml to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml_tmp
2016-03-08 22:59:19,614 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml_tmp
2016-03-08 22:59:19,645 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004.summary_tmp to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004.summary
2016-03-08 22:59:19,654 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml_tmp to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004_conf.xml
2016-03-08 22:59:19,666 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist_tmp to hdfs://C423A:8020/mr-history/tmp/hive/job_1457504560070_0004-1457506422934-hive-oozie%3Aaction%3AT%3Djava%3AW%3DTDExportMR%3AA%3Dexport%3AID%3D00001-1457506759193-0-0-FAILED-default-1457506429243.jhist
2016-03-08 22:59:19,666 INFO [Thread-89] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2016-03-08 22:59:19,671 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to Task failed task_1457504560070_0004_m_000004
Job failed as tasks failed. failedMaps:1 failedReduces:0
2016-03-08 22:59:19,672 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://apus2.labs.teradata.com:19888/jobhistory/job/job_1457504560070_0004
2016-03-08 22:59:19,680 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.
2016-03-08 22:59:20,682 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:1 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:7 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:7 ContRel:0 HostLocal:6 RackLocal:1
2016-03-08 22:59:20,684 INFO [Thread-89] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://C423A /user/hive/.staging/job_1457504560070_0004
2016-03-08 22:59:20,711 INFO [Thread-89] org.apache.hadoop.ipc.Server: Stopping server on 46067
2016-03-08 22:59:20,712 INFO [IPC Server listener on 46067] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 46067
2016-03-08 22:59:20,712 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
2016-03-08 22:59:20,714 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted.
Plese help me in resolving the issue.
You must be using sqoop to bring data to hadoop. Please paste command you are running. For "Failed to write data" , there can be multiple issues. destination parent directory is not avialble, space is not there at cluster etc.Only command can give explanation.