I have been able to setup a Nifi Registry locally and connect it to my Local Nifi Cluster.
But i have Nifi Cluster of my organization (which is on a different port) and i want to setup a Nifi Registry for it. Thus i have setup the nifi registry on a server.
Can anyone help me with procedure for doing this
Related
i am trying to run Kylo and Nifi both in cluster mode.
Nifi I am using doesn't come packaged with Kylo. I have installed nifi and kylo separately.
I am able to run Nifi in cluster mode (2 servers) and front ended it with a LB. LB has its own DNS. I am able to access it, and both nifi nodes identify each other.
I am able to install kylo in cluster mode and same way, i have front ended it with LB.
both nifi and kylo are https. I am using SAN certificate and it all is working fine.
although my infra is not internet facing (intranet), i have procured CA certs (not using self signed).
now i am trying to integrate Kylo with Nifi, so I control flows in better ways.
I am now on the step where I have to integrate kylo services with nifi.
Now issue is.
I am trying to make kylo ware of nifi, so I can use it. as per my assumption, kylo both nodes (k1 and k2) should point to LB of nifi and not directly to both nodes of nifi (n1 and n2).
i am following steps mentioned under (but not completely), as i am not relate it to my situation https://kylo.readthedocs.io/en/v0.10.0/security/ConfigureNiFiWithSSL.html
i am wondering how to generate keystore kylo service, it seems toolkit way is not working :( . it does generate the file, but still no communication with Nifi.
one strange thing, I am able to connect K1-N1 and K2-N2 (k1, k2 are 2 nodes of Kylo, n1,n2 are 2 nodes of nifi). But i am not able to connect Kylo to LB of Nifi. Unless i do that, i think I am not doing pure HA.
error i am getting on UI is, backend is not helpful either. it just shows not able to connect. although i am able to access LB dns from browser, and it works perfectly fine. =>
I want to know if I can access NiFi installed on a server from any other system. I would like to access NiFi remotely. How can I access to NiFi remotely ?
I am working with two instances of nifi.
Instance-1: A secure nifi single node.
Instance-2: A secure 3-node nifi cluster on AWS.
My site to site settings have the below configurations:
Instance-1:
nifi.remote.input.host=<hostname running locally
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
Instance-2:
nifi.remote.input.host=<ec2 public fqdn>.compute.amazonaws.com
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
My remote processor group is in locally running nifi and I am trying to push a flowfile from local to AWS cluster. I am facing error as below:
Error while trying to connect RPG
We are using cassandra enterprise editions. We have configured cluster from opscenter but we are unable to get access from Remote host. We are able to get access inside my DC from another host.
Please let me know how it can be resolved.
Note: There is no issue with firewalld.
I'm trying to use the Dedoop application that runs using Hadoop and HDFS on Amazon EC2. The Hadoop cluster is set up and the Namenode JobTracker and all other Daemons are running without error.
But the war Dedoop.war application is not able to connect to the Hadoop Namenode after deploying it on tomcat.
I have also checked to see if the ports are open in EC2.
Any help is appreciated.
If you're using Amazon AWS, I highly recommend using Amazon Elastic Map Reduce. Amazon takes care of setting up and provisioning the Hadoop cluster for you, including things like setting up IP addresses, NameNode, etc.
If you're setting up your own cluster on EC2, you have to be careful with public/private IP addresses. Most likely, you are pointing to the external IP addresses - can you replace them with the internal IP addresses and see if that works?
Can you post some lines of the Stacktrace from Tomcat's log files?
Dedoop must etablish an SOCKS proxy server (similar to ssh -D port username#host) to pass connections to Hadoop nodes on EC2. This is mainly because Hadoop resolves puplic IPS to EC2-internal IPs which breaks MR Jobs submission and HDFS management.
To this end Tomcat must be configured to to etablish ssh connections. The setup procedure is described here.