Unable to get NiFi to work in cluster - cluster-computing

Question

Based on all the examples I've read I thought setting up NiFi as a cluster would be easy. Apparently I can't get it to work. I'm using NiFi 1.5. I only have 2 host and pretended that there was a third, but NiFi is not starting as a cluster. These are the changes I've made to the config files.
state-management.xml file
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect String">etl-1:2181,etl-2:2181,etl-3:2181</property>
<property name="Root Node">/ssd/nifi-1.5.0</property>
<property name="Session Timeout">10 seconds</property>
<property name="Access Control">Open</property>
</cluster-provider>
zookeeper.properities file
dataDir=./state/zookeeper
server.1=etl-1:2888:3888
server.2=etl-2:2888:3888
server.3=etl-3:2888:3888
nifi.properties file
nifi.zookeeper.connect.string=etl-1:2181,etl-2:2181,etl-3:2181
nifi.zookeeper.root.node=/ssd/nifi-1.5.0
nifi.cluster.is.node=yes
nifi.cluster.node.address=etl-2 (this is set to etl-1 on the other node)
nifi.state.management.embedded.zookeeper.start=true
[nifi#etl-2 zookeeper]$ cat /ssd/nifi-1.5.0/state/zookeeper/myid
2
[nifi#etl-1 logs]$ cat /ssd/nifi-1.5.0/state/zookeeper/myid
1
I've updated logback.xml to DEBUG, but there are so many message I can't seem to find out's what wrong. My best guess is that zookeeper in starting in local instead of cluster.
ls -l /ssd/nifi-1.5.0/state/
total 4
drwxrwxr-x. 18 nifi nifi 4096 Feb 23 18:49 local
drwxrwxr-x. 2 nifi nifi 18 Feb 23 15:26 zookeeper

Answer 1

I found the problem. I had nifi.cluster.is.node=yes instead of nifi.cluster.is.node=true.

Answer 2

Am trying to run Nifi as a 3 node cluster on Windows 10, with same properties set in all 3 conf files as mentioned above, but still its failing to start nifi as a 3 node cluster.
Below are the settings done on my files
state-management.xml file
<cluster-provider>
<id>zk-provider</id>
<class>org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider</class>
<property name="Connect String">node1:2181,node2:2181,node3:2181</property>
<property name="Root Node">/nifi</property>
<property name="Session Timeout">10 seconds</property>
<property name="Access Control">Open</property>
</cluster-provider>
zookeper properties
dataDir=./state/zookeeper
autopurge.snapRetainCount=30
server.1=node1:2888:3888
server.2=node2:2888:3888
server.3=node3:2888:3888
nifi properties:
nifi.web.http.host=node1
nifi.web.http.port=8080
nifi.cluster.is.node=true
nifi.cluster.node.address=node1
nifi.cluster.node.protocol.port=11122
nifi.zookeeper.connect.string=node1:2181,node2:2181,node3:2181
nifi.zookeeper.root.node=/nifi
In other 2 noes, the host name are placed respectively
But when i start the nodes, it displays the below error
C:\nifi-1.11.0-bin\Node1\bin>run-nifi.bat
2020-02-11 20:29:20,237 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi...
2020-02-11 20:29:20,238 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: C:\NIFI-1~1.0-B\Node1
2020-02-11 20:29:20,239 INFO [main] org.apache.nifi.bootstrap.Command Command: C:\Program Files\Java\jdk1.8.0_241\bin\java.exe -classpath C:\NIFI-1~1.0-B\Node1.\conf;C:\NIFI-1~1.0-B\Node1.\lib\javax.servlet-api-3.1.0.jar;C:\NIFI-1~1.0-B\Node1.\lib\jcl-over-slf4j-1.7.26.jar;C:\NIFI-1~1.0-B\Node1.\lib\jetty-schemas-3.1.jar;C:\NIFI-1~1.0-B\Node1.\lib\jul-to-slf4j-1.7.26.jar;C:\NIFI-1~1.0-B\Node1.\lib\log4j-over-slf4j-1.7.26.jar;C:\NIFI-1~1.0-B\Node1.\lib\logback-classic-1.2.3.jar;C:\NIFI-1~1.0-B\Node1.\lib\logback-core-1.2.3.jar;C:\NIFI-1~1.0-B\Node1.\lib\nifi-api-1.11.0.jar;C:\NIFI-1~1.0-B\Node1.\lib\nifi-framework-api-1.11.0.jar;C:\NIFI-1~1.0-B\Node1.\lib\nifi-nar-utils-1.11.0.jar;C:\NIFI-1~1.0-B\Node1.\lib\nifi-properties-1.11.0.jar;C:\NIFI-1~1.0-B\Node1.\lib\nifi-runtime-1.11.0.jar;C:\NIFI-1~1.0-B\Node1.\lib\slf4j-api-1.7.26.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djavax.security.auth.useSubjectCredsOnly=true -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dzookeeper.admin.enableServer=false -Dnifi.properties.file.path=C:\NIFI-1~1.0-B\Node1.\conf\nifi.properties -Dnifi.bootstrap.listen.port=53535 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=C:\NIFI-1~1.0-B\Node1\bin..\logs org.apache.nifi.NiFi
2020-02-11 20:29:20,554 WARN [main] org.apache.nifi.bootstrap.Command Failed to set permissions so that only the owner can read pid file C:\NIFI-1~1.0-B\Node1\bin..\run\nifi.pid; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2020-02-11 20:29:20,561 WARN [main] org.apache.nifi.bootstrap.Command Failed to set permissions so that only the owner can read status file C:\NIFI-1~1.0-B\Node1\bin..\run\nifi.status; this may allows others to have access to the key needed to communicate with NiFi. Permissions should be changed so that only the owner can read this file
2020-02-11 20:29:20,573 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 4284
C:\nifi-1.11.0-bin\Node1\bin>status-nifi.bat
20:31:57.265 [main] DEBUG org.apache.nifi.bootstrap.NotificationServiceManager - Found 0 service elements
20:31:57.271 [main] INFO org.apache.nifi.bootstrap.NotificationServiceManager - Successfully loaded the following 0 services: []
20:31:57.272 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_STARTED
20:31:57.273 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_STOPPED
20:31:57.274 [main] INFO org.apache.nifi.bootstrap.RunNiFi - Registered no Notification Services for Notification Type NIFI_DIED
20:31:57.277 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File: C:\NIFI-1~1.0-B\Node1\bin..\run\nifi.status
20:31:57.278 [main] DEBUG org.apache.nifi.bootstrap.Command - Status File: C:\NIFI-1~1.0-B\Node1\bin..\run\nifi.status
20:31:57.279 [main] DEBUG org.apache.nifi.bootstrap.Command - Properties: {pid=4284, port=53536}
20:31:57.280 [main] DEBUG org.apache.nifi.bootstrap.Command - Pinging 53536
20:31:58.343 [main] DEBUG org.apache.nifi.bootstrap.Command - Process with PID 4284 is not running
20:31:58.345 [main] INFO org.apache.nifi.bootstrap.Command - Apache NiFi is not running