I want to submit spark jobs from NIFI using Executivespark interactive processor, for that I am setting up livysession controller service.
The ip and port names are correct, when I try to connect it's throwing error as shown in the attachment can anyone help me configure the controller service ?
ERROR: Timeout connecting to Livy Service to retrieve sessions:java.net.SocketTimeoutException: Read Time Out
LivySessionControllerService
Related
I start my spring boot app like:
ExecStart=/bin/java -javaagent:./opentelemetry-javaagent.jar -jar /myapp/app.jar -Xmx4gi -Dotel.traces.exporter=otlp -Dotel.javaagent.debug=true -Dotel.metrics.exporter=logging -Dotel.exporter.otlp.endpoint=http://GRAFANA_TEMPO_IP:4317 -Dotel.exporter.otlp.traces.endpoint=GRAFANA_TEMPO_IP:4317 -Dotel.resource.attributes=service.name=app
Why it still sends traffic to: The request could not be executed. Full error message: Failed to connect to localhost/127.0.0.1:4317? Why it send to localhost if variables are defined when starting java app?
Do I need in this case collector from Opentelemetry, or can I use only an agent?
I'd like to let nifi eat its own bulletin message by using the "SiteToSiteBulletinReportingTask" controller. here is my setting nifi s2s bulletinReportingTask
however although nifi instance can eat the bulletin message but it always raise warning massage like:
23:47:37 CSTWARNINGdf68bf40-016e-1000-b458-a0a98bf3ebc6
SiteToSiteBulletinReportingTask[id=df68bf40-016e-1000-b458-a0a98bf3ebc6] org.apache.nifi.remote.client.PeerSelector#2ae7fbef Unable to refresh Remote Group's peers due to null
Any advice?
PS.
there is "state" folder under "nifi/" and i also manually created "state" folder under "nifi/conf/"
This is a known issue here --- https://issues.apache.org/jira/browse/NIFI-6886
INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is currently running, listening to Bootstrap on port 20740, PID=31405
That means NiFi is running, and I can see its log in logs/nifi-app.log.
The UI is available at https://servername:9443/nifi. It successfully initiated communication with Bootstrap.
But i have this error, and the UI fails to appear. Do you have any solutions?
Failed to invoke #OnEnabled method of JettyWebSocketServer[id=01591009-1d2b-177f-e304-a7cc87d735ce] due to java.net.BindException: Address already in use
"java.net.BindException: Address already in use" means a port is already in use by something else on your system.
In this case it shows that it is coming from the JettyWebSocketServer controller service which is trying to bind to port 9998.
You can use "netstat -lntp" to see what is using port 9998. It could also be possible that you have more than one instance of the controller service with both of them configured with port 9998 and one of them is binging to it and the other fails.
Seems like a Controller Service that fails... Try browsing your flow.xml.gz and check for any ControllerServices named JettyWebSocketServer. Change their <state></state> from <state>ENABLED</state> to <state>DISABLED</state> and try running NiFi again.
You can tell it is a controller service by the #enabledannotation which invokes a method that needs to run when the controller service is enabled.
NiFi shouldn't fail to start because of a failing controller service but it seems to be the problem(probably a bug).
After it start back up, you can configure the controller service to run on a different port.
I also suggest, that if it is the case, that you open an issue to Apache about this since it seems like a pretty major bug.
I have build a simple Process Group. It generates a FlowFile with some random stuff in it and sends it to the Nifi Remote Process Group.
This Remote Process Group is configured to send the FlowFile to localhost or in this case to my own Hostname (I have tried localhost as well).
After this the FlowFile should Appear at the "From MiNiFi" input Port and is sended to the LogAttribute. Nothing Special.
I configured to using RAW but with HTTP it neither works.
I am using the apache/nifi docker image and didn´t changed something in nifi.properties and authorizers.xml but of couse i provide you both:
nifi.properties
authorizers.xml
The Error occuring is this:
WARNING org.apache.nifi.remote.client.PeerSelector#40081613 Unable to refresh Remote Group´s peers due to Unable to communicate with remote Nifi cluster in order to determine which nodes exist in the remote cluster
I hope you can help me. I have wasted too much time with this Problem XD
In nifi.properties you have nifi.web.http.host=f4f40c87b65f so that means the hostname that NiFi is listening for requests on is f4f40c87b65f which means the URL of your RPG must be http://f4f40c87b65f:8080/nifi
I have installed HDF 2.1.2 with 3 node Nifi 1.1 cluster and I am trying to configure controller service in it. I created a simple ExecuteHQL processor which depends on ThriftConnectionPool controller service.
I dragged ExecuteProcessor on canvas and as soon as I clicked on gear icon for configuring controller service , it throws runtime exception in log.
Attempting request for (anonymous) GET http://<hostname>:9090/nifi-api/flow/process-groups/83490f62-015c-1000-0000-00004da8f033/controller-
HTTP 500 Internal Server Error
I had duplicate controller service jar - one actual and another symlink which was causing issue.