apache nifi: hive closes connection and workflows stop working - apache-nifi

I noticed some workflows stop working because the connection pool looses connection to hive, if I stop and then restart the connection pool anything work. How can I avoid this problem working only on hive? Does anybody faced this problem ?

This is a bug, DBCP currently assumes the connections are valid, so it can return such a connection the next time the processor runs, causing the error you are seeing.
I think HiveConnectionPool would benefit from the same fix in NIFI-2381, and perhaps other DBCP settings to ensure only valid connections are returned/borrowed. I have written NIFI-2927 to cover this.
Unfortunately I believe the only workaround at present is to restart NiFi.

Related

MySql server has gone away when run in queue

We have maradb 10.1 and beanstalkd 1.10 and laravel 4.2
We have one query that run successfully without queue. but when run it in beanstalkd not afected and we get 'MySql server has gone away' error in log file
config:
wait_timeout = 120
max_allowed_packet = 1024M
Why different behavior between with and without queue
We had similar issues and either it was because the code was running in different thread, and connection being lost, or a strange garbage collection and closing of connection for long running processes.
Anyway what we implemented is
- when a job is reserved and starts processing we reconnect the DB always
- when we detect a connection gone away, we release the job (so it will be picked up again)
In case it happens in the middle of the processing flow, you may want to reconnect to lose work done so far on that job, if the job is somehow transactional.

How I can keep and restore JMSservers to their last known state after a reboot

I am trying to find if there is any parameter which will keep the Consumption Parameters as the previous workable state after a full restart of Weblogic Server.
I would like to restore JMSservers to their last known state after a reboot-restart.
I have tested and i have enabled the ConsumptionPaused to true in a JMSServer via wlst and I have done a full restart but after the restarting the ConsumptionPaused has changed to false.
Please for your help.
I think you can achieve this using the setConsumptionPausedAtStartup method on the JMSServerMBean mbean
Finally, I have found that with the parameter ConsumptionPausedAtStartup enabled, after the restart of Weblogic then all the osb resources under the destination of JMSServer will have the ConsumptionPaused as true in JMS Queue layer level and not in JMS Server layer.
If you will not disabled the parameter ConsumptionPausedAtStartup, then every time you will restart the server then all the queues under this JMSServer will be with ConsumptionPaused as true.
Best Regards,
Nickname

How do I prevent a database call from hanging in jdbcTemplate.query when the database is not available?

I am using the Spring JDBC template and a BoneCP Connection Pool. When I purposely set the JDBC URL to an invalid value (to test how my system works on failed database connectivity), it throws an UnknownHostException on server start-up, but the server continues to start. When I submit a request, the jdbcTemplate.query(sql) method hangs. What is a good way to handle this so the system doesn't hang in jdbcTemplate.query(sql)?
This scenario is possible if the database is down due to network connectivity issues. I tried to play around with the idleMaxAgeInSeconds and maxConnectionAgeInSeconds values; I set them both to 10, but the code still hangs in jdbcTemplate.query().
I'm not sure I understand your question, but it seems to me either you don't want the server to start if the exception occurs, or you want to set a static variable that checks if the database is on before trying to contact it (no UnknownHostException when initializing jdbc).
Maybe this points you towards your answer:
http://www.cubrid.org/blog/dev-platform/understanding-jdbc-internals-and-timeout-configuration/
good luck!

Unable to connect to apache solr - process restart needed

I am facing a strange problem with solr. After running solr for few hours, client starts reporting error message that it is unable to contact the solr, although solr instance is up on the server.
I can't see any high traffic on website which sometimes is the reason of connection refusal.
This issue gets fixed after solr restart.
Any idea what is going wrong here ?
Answer to most of the problems can be found in logs. Thanks D_K for reminding me.
SEVERE: java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
I have increased the heap size to fix this issue.
java -Xms<initial heap size> -Xmx<maximum heap size>
Also, we have reduced document size by removing unnecessary information which we don't need to retain in solr.
If you have a client with long running connection but low amount of traffic, you may have a firewall in between. Firewalls have limited-size routing tables, so they eventually drop the mapping for connections they haven't seen for a while.
Try sending a ping query every 30 minutes or so through that specific connection and see if the issue goes away. If you need to validate it, run Wireshark on the client and see whether the client is getting RST (reset) packets from an unexpected end point (that would be firewall).
You just need to add your collection in solr by following the step given in this url ( https://drupal.stackexchange.com/questions/95897/apache-solr-4-6-0-insta... ) and then select your collection from your solr which is running on localhost or live site (http://localhost:8983/solr/) and go to schema tab. Click schema tab and then you can see you schema file attach in apachesolr module.
You now just need to your schema url which just look like this http://localhost:8983/solr/your_core_name/. Now add this url in apachesolr module.
Then it will show that your site has contacted apache solr server in your drupal site.

JDBC URL for Oracle XA client

Using the JDBC driver oracle.jdbc.xa.client.OracleXADataSource, what is the correct format of the JDBC URL? The thin format of
jdbc:oracle:thin:#host:port:sid
does not work. WebSphere is reporting that the given url (which is otherwise correct) is invalid.
The test connection operation failed for data source Oracle MyDB (XA) on
server nodeagent at node MY_node with the following exception:
java.sql.SQLException: Invalid Oracle URL specifiedDSRA0010E: SQL State = 99999,
Error Code = 17,067. View JVM logs for further details.
There is nothing in the JVM logs.
Whether you use a XA Driver or not, the JDBC connection string is the same (and the format of your question is correct).
For me the issue resolved by adding alias name, username and password in JAAS - J2C authentication data. And also selecting this entry as Component-managed authentication alias.
In case this happens to anyone else. The problem went away after restarting websphere.
In my case, the problem went away when I change the authentication property of the jdbc resource reference from Authentication=Application to Authentication=Container
Had the same issue. Dont know about simple deployments, but on a two nodes cluster, I restarted the first node, and the connection started working on it (not on the second). Restarted the second node, and the connection started working there too.
So just restart the nodes (I also restarted the nodeAgents, but i don't know if it's necessary).
if you are doing using wsadmin command then you need to stop manager,stop node,start manager, sync node and then start node (I mean full sync). Hopefully this will resolve the issue. I dont know why but this resolves my issue.

Resources