I downloaded cassandra from the official site and ran it with:
./bin/cassandra -f
Cassandra seems to be working fine and I am able to connect to it via cqlsh
When I stop it using CTRL-C, it throws an error.
INFO 17:39:06 Stop listening to thrift clients
INFO 17:39:06 Stop listening for CQL clients
INFO 17:39:06 Announcing shutdown
INFO 17:39:06 Node localhost/127.0.0.1 state jump to normal
INFO 17:39:08 Waiting for messaging service to quiesce
INFO 17:39:08 MessagingService has terminated the accept() thread
ERROR 17:39:08 Exception in thread Thread[StorageServiceShutdownHook,5,main]
java.io.IOError: java.io.IOException: Unknown error: 316
at org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:750) ~[apache-cassandra-2.1.9.jar:2.1.9]
at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:682) ~[apache-cassandra-2.1.9.jar:2.1.9]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.1.9.jar:2.1.9]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_60]
Caused by: java.io.IOException: Unknown error: 316
at sun.nio.ch.NativeThread.signal(Native Method) ~[na:1.8.0_60]
at sun.nio.ch.ServerSocketChannelImpl.implCloseSelectableChannel(ServerSocketChannelImpl.java:292) ~[na:1.8.0_60]
at java.nio.channels.spi.AbstractSelectableChannel.implCloseChannel(AbstractSelectableChannel.java:234) ~[na:1.8.0_60]
at java.nio.channels.spi.AbstractInterruptibleChannel.close(AbstractInterruptibleChannel.java:115) ~[na:1.8.0_60]
at sun.nio.ch.ServerSocketAdaptor.close(ServerSocketAdaptor.java:137) ~[na:1.8.0_60]
at org.apache.cassandra.net.MessagingService$SocketThread.close(MessagingService.java:1017) ~[apache-cassandra-2.1.9.jar:2.1.9]
at org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:746) ~[apache-cassandra-2.1.9.jar:2.1.9]
... 3 common frames omitted
I was wondering whether I missed any setup procedures. The getting started guide says it should running out of the box. I am using OSX Yosemite 10.10.5
Thanks in advance.
It appears this is a JDK bug and it looks like this could be fixed by updating your JDK, what version are you currently on?
It looks like CASSANDRA-8220 (C* 2.2.1+, and 3.0.0-alpha1) was introduced to work around the problem, but I think upgrading your JDK should fix this as well.
Related
When running thrift (/usr/hdp/2.3.0.0-2557/hbase/bin/hbase-daemon.sh start thrift) it stops working every once in a while.
In the logs I can see the exception:
2015-11-12 11:56:11,926 WARN [thrift-worker-3] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x20ae27f0 closed
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
... 22 more
I end up restarting thrift to get it working again.
I've tried implementing the following workaround found here but its still crashing:
<property>
<name>hbase.thrift.connection.max-idletime</name>
<value>1800000</value>
</property>
We are using HDP 2.3.0 stack from HortonWorks / HBase 1.1.1.2.3.0.0-2557
This is an ongoing issue with HBase. There are two seperate tickets regarding this problem: HBASE-14533 and HBASE-14196. Both tickets provide a patch to resolve the problem.
Regarding to Pankaj Kumar's comment from 01/19 the issue can be resolved if you merge both patches. Maybe this helps in your case, too.
When a .wlapp file is deployed in MobileFirst Console, we get the following error in the logs. This happened after we upgraded MobileFirst Server from 6.3 to 7, but don't know if this is related to the version of MobileFirst.
000001c6 BaseTransacti E RuntimeMBeanCallable.call() exception
java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy166.deployApplication(Unknown Source)
at com.ibm.worklight.admin.actions.ApplicationDeploymentTransaction.prepareMBean(ApplicationDeploymentTransaction.java:919)
at com.ibm.worklight.admin.actions.util.RuntimeMBeanWorkerThreadCaller$RuntimeMBeanCallable.call(RuntimeMBeanWorkerThreadCaller.java:76)
at com.ibm.worklight.admin.actions.util.RuntimeMBeanWorkerThreadCaller.callSynchronously(RuntimeMBeanWorkerThreadCaller.java:183)
at com.ibm.worklight.admin.actions.util.RuntimeMBeanPoolCaller.callRuntimeMBeans(RuntimeMBeanPoolCaller.java:91)
at com.ibm.worklight.admin.actions.BaseTransaction.prepare(BaseTransaction.java:450)
at com.ibm.worklight.admin.actions.BaseTransaction.internalRun(BaseTransaction.java:348)
at com.ibm.worklight.admin.actions.BaseTransaction$1.run(BaseTransaction.java:235)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:790)
Caused by: com.ibm.websphere.management.exception.ConnectorException: ADMC0009E: The system failed to make the SOAP RPC call: invoke
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplateOnce(SOAPConnectorClient.java:894)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplate(SOAPConnectorClient.java:689)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplate(SOAPConnectorClient.java:679)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invoke(SOAPConnectorClient.java:665)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invoke(SOAPConnectorClient.java:487)
at com.sun.proxy.$Proxy121.invoke(Unknown Source)
at com.ibm.ws.management.AdminClientImpl.invoke(AdminClientImpl.java:224)
at com.worklight.common.util.jmx.WASRuntimeMBeanHandler$AdminClientMBeanServerConnection.invoke(WASRuntimeMBeanHandler.java:521)
at com.sun.jmx.mbeanserver.MXBeanProxy$InvokeHandler.invoke(MXBeanProxy.java:146)
at com.sun.jmx.mbeanserver.MXBeanProxy.invoke(MXBeanProxy.java:160)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:246)
... 11 more
Caused by: [SOAPException: faultCode=SOAP-ENV:Client; msg=Read timed out; targetException=java.net.SocketTimeoutException: Read timed out]
at org.apache.soap.transport.http.SOAPHTTPConnection.send(SOAPHTTPConnection.java:479)
at org.apache.soap.rpc.Call.WASinvoke(Call.java:510)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient$8.run(SOAPConnectorClient.java:852)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.connector.soap.SOAPConnectorClient.invokeTemplateOnce(SOAPConnectorClient.java:845)
... 21 more
MobileFirst version 7
WAS server version 8.5.5.5
JRE version 1.6
Re-deploying the MobileFirst artifacts to the application server should normally not be required as inspecting the logs and configuration usually will lead to the root cause of the problem. However in some cases starting over to get things right will work.
As mentioned in the comments, re-deployment of the server artifacts nullified the issue that was experienced.
Gradle throws exception with message "Unable to establish loopback connection". The following is the stack trace thrown at the console.
java.io.IOException: Unable to establish loopback connection
org.gradle.internal.UncheckedException: java.io.IOException: Unable to establish loopback connection
at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:39)
at org.gradle.messaging.remote.internal.inet.SocketConnection.<init>(SocketConnection.java:58)
at org.gradle.messaging.remote.internal.inet.SocketConnectCompletion.create(SocketConnectCompletion.java:43)
at org.gradle.messaging.remote.internal.hub.MessageHubBackedObjectConnection.connect(MessageHubBackedObjectConnection.java:92)
at org.gradle.api.internal.tasks.testing.worker.ForkingTestClassProcessor.forkProcess(ForkingTestClassProcessor.java:78)
at org.gradle.api.internal.tasks.testing.worker.ForkingTestClassProcessor.processTestClass(ForkingTestClassProcessor.java:56)
at org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor.processTestClass(RestartEveryNTestClassProcessor.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.messaging.dispatch.FailureHandlingDispatch.dispatch(FailureHandlingDispatch.java:29)
at org.gradle.messaging.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:132)
at org.gradle.messaging.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:33)
at org.gradle.messaging.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:72)
at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Gradle version :
Gradle 1.12
Build time: 2014-04-29 09:24:31 UTC
Build number: none
Revision: a831fa866d46cbee94e61a09af15f9dd95987421
Groovy: 1.8.6
Ant: Apache Ant(TM) version 1.9.3 compiled on December 23 2013
Ivy: 2.2.0
JVM: 1.7.0_45 (Oracle Corporation 24.45-b08)
OS: Windows 7 6.1 x86
Jdk is 1.7.45.
Is there any help? I tried stopping the anti virus assuming the connection are getting terminated by AV. However, the issue re-occurred when the AV was turned off.
I assume this is a problem where Gradle cannot connect to the gradle Daemon. Your machine probably has an issue with its networking config, which needs to be worked out.
A possible workaround would be to run without the Daemon, using the --no-daemon option. If that still doesn't work, there is some other reason why it tries to establish a connection, probably due to a custom plugin or build script.
I was using JDK v1.7.0_45 and I now use 1.7.0_79. Ever since the upgrade I am not observing this issue. I switched back to v1.7.0_45 this morning and I am already noticed "Unable to establish loopback connection" twice out of six iterations.
I'm trying to run the distributed shell example (using a SVN checkout of Hadoop, which is why the version is set to 3.0.0-SNAPSHOT):
yarn jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar \
-jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar \
org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command whoami
However it does not work:
12/09/03 13:44:37 FATAL distributedshell.Client: Error running CLient
java.lang.reflect.UndeclaredThrowableException
at org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:128)
at org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getClusterMetrics(ClientRMProtocolPBClientImpl.java:123)
at org.hadoop.yarn.client.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:163)
at org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:316)
at org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:164)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.getProtocolImpl(ProtobufRpcEngine.java:398)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:456)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1732)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1728)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1726)
at org.apache.hadoop.ipc.Client.call(Client.java:1164)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
at $Proxy7.getClusterMetrics(Unknown Source)
at org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getClusterMetrics(ClientRMProtocolPBClientImpl.java:121)
... 8 more
The essential problem seems to be in the second trace:
Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB
Does anyone know how protocol registration for Hadoops ProtoBufRPC works? Any idea on how to debug?
Edit: With Hadoop version 2.0.1-alpha, it works slightly better.
12/09/03 18:43:14 INFO distributedshell.Client: Application did not finish. YarnState=FAILED, DSFinalStatus=FAILED. Breaking monitoring loop
12/09/03 18:43:14 ERROR distributedshell.Client: Application failed to complete successfully
So maybe my build did not work right. Any ideas of what is causing the problem above (I'd really like to use HEAD, as I'm planning to do some low level experiments, beyond MapReduce)? Or is HEAD partially broken, does distributed shell on HEAD work for you?
My own (not yet working ...) client still fails with the same error:
Caused by: java.io.IOException: Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB
It turned out that the main problem with my own code was that I naively instantiated the Configuration class, instead of instantiating YarnConfiguration. This way, the yarn config files were not read, and it tried to contact the servers on their default ports - which don't agree with my settings.
The same bug seems to be present in the distributedshell example.
I am trying to run my application via Netbeans 7.0 on tomcat 7.0.11 (bundled with the Netbeans itself).
When I try to run the application I am getting the following error.
I have reinstalled everything many times but I am still getting same issues.
It runs properly first time and then if I stop the server and then try to start it again then it creates the problem. At times when I restart my PC it works and sometimes even restarting the PC does not work.
Below is the error stack from Netbeans 7.0
init:
deps-module-jar:
deps-ear-jar:
deps-jar:
library-inclusion-in-archive:
library-inclusion-in-manifest:
compile:
compile-jsps:
Checking data source definitions for missing JDBC drivers...
Starting Tomcat process...
Waiting for Tomcat...
Starting of Tomcat failed.
E:\workspaces\netbeansws\myApp\nbproject\build-impl.xml:770:
Deployment error:
Starting of Tomcat failed.
See the server log for details.
at org.netbeans.modules.j2ee.deployment.devmodules.api.Deployment.deploy(Deployment.java:200)
at org.netbeans.modules.j2ee.ant.Deploy.execute(Deploy.java:106)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
at org.apache.tools.ant.module.bridge.impl.BridgeImpl.run(BridgeImpl.java:284)
at org.apache.tools.ant.module.run.TargetExecutor.run(TargetExecutor.java:539)
at org.netbeans.core.execution.RunClassThread.run(RunClassThread.java:154)
BUILD FAILED (total time: 3 minutes 0 seconds)
Please help !
I'd recommend looking at the the Tomcat logs to find out what the specific error is.