Related
I have a Go service doing heavy reads from an NFS / GPFS volume. I do occasionally run into issues at scale during which the underlying mount would not answer to a specific syscall, resulting in the entire service being taken down by the kernel:
[98549.941930] Tainted: G O 4.14.13-1.el7.elrepo.x86_64 #1
[98549.942454] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[98549.943422] ls D 0 14884 1 0x00000084
[98549.943968] Call Trace:
[98549.944498] __schedule+0x28d/0x880
[98549.945033] schedule+0x36/0x80
[98549.945552] schedule_preempt_disabled+0xe/0x10
[98549.946095] __mutex_lock.isra.5+0x269/0x500
[98549.946611] __mutex_lock_slowpath+0x13/0x20
[98549.947153] mutex_lock+0x2f/0x40
[98549.947695] fuse_lock_inode+0x2a/0x30 [fuse]
[98549.948248] fuse_readdir+0x113/0x7e0 [fuse]
[98549.948795] iterate_dir+0x16e/0x190
[98549.949323] ? __audit_syscall_entry+0xaf/0x100
[98549.949847] SyS_getdents+0x98/0x120
[98549.950358] ? iterate_dir+0x190/0x190
[98549.950898] do_syscall_64+0x67/0x1b0
[98549.951410] entry_SYSCALL64_slow_path+0x25/0x25
[98549.951948] RIP: 0033:0x7ffff749dcb5
[98549.952454] RSP: 002b:00007fffffffd160 EFLAGS: 00000246 ORIG_RAX: 000000000000004e
[98549.953423] RAX: ffffffffffffffda RBX: 00000000006260a0 RCX: 00007ffff749dcb5
[98549.953985] RDX: 0000000000008000 RSI: 00000000006260a0 RDI: 0000000000000005
[98549.954518] RBP: 00000000006260a0 R08: 0000000000000080 R09: 0000000000008030
[98549.955131] R10: 00007fffffffced0 R11: 0000000000000246 R12: fffffffffffffe90
[98549.955655] R13: 0000000000000000 R14: 0000000000626030 R15: 0000000000626000
I an looking for a way to add a timeout so that no individual failed syscall can take the whole service down, but cannot find a good way to do it in Go.
One usual design I found is to run syscalls from an OS thread and kill this thread on timeout, but that does not seem like a possibility with Golang due to the lack of control on the underlying system threads. The service is typically executing large amounts of syscalls in parallel (possibly hundreds).
you can get the pid of the process using the sycall package
pid, _, _ := syscall.Syscall(syscall.SYS_GETPID, 0, 0, 0)
and latter kill the process by pid
select {
case end = <-endSignal:
fmt.Println("The end!")
case <-time.After(5 * time.Second):
proc, _ := os.FindProcess(pid)
// Kill the process
proc.Kill()
}
We are using spring-websocket with WebSphere 16.0.0.3
And Time after time we are facing issue when all clientOutboundChannelExecutor are stuck.
3XMTHREADINFO "clientOutboundChannel-1" J9VMThread:0x0000000001834900, j9thread_t:0x00007F2EDEFD1030, java/lang/Thread:0x00000000E2E30A28, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x44, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0xDF6A, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x01000201)
3XMTHREADINFO2 (native stack address range from:0x00007F2EDE11A000, to:0x00007F2EDE15B000, size:0x41000)
3XMCPUTIME CPU usage total: 49.720264588 secs, current category="Application"
3XMTHREADBLOCK Blocked on: java/lang/Object#0x00000000E30B30F8 Owned by: "clientOutboundChannel-4" (J9VMThread:0x00000000019CEE00, java/lang/Thread:0x00000000E2E30C98)
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.resetRequest(AbstractHttpSockJsSession.java:334(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.disconnect(AbstractHttpSockJsSession.java:330(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractSockJsSession.close(AbstractSockJsSession.java:216(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/WebSocketSessionDecorator.close(WebSocketSessionDecorator.java:158(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/ConcurrentWebSocketSessionDecorator.close(ConcurrentWebSocketSessionDecorator.java:192(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.sendToClient(StompSubProtocolHandler.java:457(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.handleMessageToClient(StompSubProtocolHandler.java:427(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/SubProtocolWebSocketHandler.handleMessage(SubProtocolWebSocketHandler.java:338(Compiled Code))
3XMTHREADINFO "clientOutboundChannel-2" J9VMThread:0x00000000019A5A00, j9thread_t:0x00007F2EDEFD15B0, java/lang/Thread:0x00000000E2E30AF8, state:CW, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x45, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0xDF6B, native priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x01000101)
3XMTHREADINFO2 (native stack address range from:0x00007F2EDDE5F000, to:0x00007F2EDDEA0000, size:0x41000)
3XMCPUTIME CPU usage total: 0.281253777 secs, current category="Application"
3XMTHREADBLOCK Waiting on: com/ibm/ws/tcpchannel/internal/SimpleSync#0x00000000E3155FC8 Owned by: <unowned>
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at java/lang/Object.wait(Native Method)
4XESTACKTRACE at java/lang/Object.wait(Object.java:172)
4XESTACKTRACE at com/ibm/ws/tcpchannel/internal/SimpleSync.simpleWait(SimpleSync.java:35)
5XESTACKTRACE (entered lock: com/ibm/ws/tcpchannel/internal/SimpleSync#0x00000000E3155FC8, entry count: 1)
4XESTACKTRACE at com/ibm/ws/tcpchannel/internal/NioTCPWriteRequestContextImpl.processSyncWriteRequest(NioTCPWriteRequestContextImpl.java:77(Compiled Code))
4XESTACKTRACE at com/ibm/ws/tcpchannel/internal/TCPWriteRequestContextImpl.write(TCPWriteRequestContextImpl.java:119(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/HttpServiceContextImpl.synchWrite(HttpServiceContextImpl.java:2438(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/HttpServiceContextImpl.sendOutgoing(HttpServiceContextImpl.java:2174(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/inbound/HttpInboundServiceContextImpl.sendResponseBody(HttpInboundServiceContextImpl.java:753(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/outbound/HttpOutputStreamImpl.flushBuffers(HttpOutputStreamImpl.java:481(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/outbound/HttpOutputStreamImpl.flush(HttpOutputStreamImpl.java:579(Compiled Code))
4XESTACKTRACE at com/ibm/wsspi/http/ee7/HttpOutputStreamEE7.flush(HttpOutputStreamEE7.java:293(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer/osgi/response/WCOutputStream.flush(WCOutputStream.java:234(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer31/osgi/response/WCOutputStream31.flush(WCOutputStream31.java:599(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer/srt/SRTServletResponse.flushBuffer(SRTServletResponse.java:705(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer/srt/SRTServletResponse.flushBuffer(SRTServletResponse.java:670)
4XESTACKTRACE at org/springframework/http/server/ServletServerHttpResponse.flush(ServletServerHttpResponse.java:96(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.writeFrameInternal(AbstractHttpSockJsSession.java:365)
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractSockJsSession.writeFrame(AbstractSockJsSession.java:322(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/StreamingSockJsSession.flushCache(StreamingSockJsSession.java:86(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.sendMessageInternal(AbstractHttpSockJsSession.java:302(Compiled Code))
5XESTACKTRACE (entered lock: java/lang/Object#0x00000000E33995A8, entry count: 1)
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractSockJsSession.sendMessage(AbstractSockJsSession.java:166(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/ConcurrentWebSocketSessionDecorator.tryFlushMessageBuffer(ConcurrentWebSocketSessionDecorator.java:132(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/ConcurrentWebSocketSessionDecorator.sendMessage(ConcurrentWebSocketSessionDecorator.java:104(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.sendToClient(StompSubProtocolHandler.java:440(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.handleMessageToClient(StompSubProtocolHandler.java:427(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/SubProtocolWebSocketHandler.handleMessage(SubProtocolWebSocketHandler.java:338(Compiled Code))
3XMTHREADINFO "clientOutboundChannel-3" J9VMThread:0x00000000019C9D00, j9thread_t:0x00007F2EDEFDAE00, java/lang/Thread:0x00000000E2E30BC8, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x46, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0xDF6C, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x01000201)
3XMTHREADINFO2 (native stack address range from:0x00007F2EDDE1E000, to:0x00007F2EDDE5F000, size:0x41000)
3XMCPUTIME CPU usage total: 42.712892035 secs, current category="Application"
3XMTHREADBLOCK Blocked on: java/lang/Object#0x00000000E33995A8 Owned by: "clientOutboundChannel-2" (J9VMThread:0x00000000019A5A00, java/lang/Thread:0x00000000E2E30AF8)
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.resetRequest(AbstractHttpSockJsSession.java:334(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.disconnect(AbstractHttpSockJsSession.java:330)
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractSockJsSession.close(AbstractSockJsSession.java:216(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/WebSocketSessionDecorator.close(WebSocketSessionDecorator.java:158)
4XESTACKTRACE at org/springframework/web/socket/handler/ConcurrentWebSocketSessionDecorator.close(ConcurrentWebSocketSessionDecorator.java:192)
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.sendToClient(StompSubProtocolHandler.java:457(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.handleMessageToClient(StompSubProtocolHandler.java:427(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/SubProtocolWebSocketHandler.handleMessage(SubProtocolWebSocketHandler.java:338(Compiled Code))
3XMTHREADINFO "clientOutboundChannel-4" J9VMThread:0x00000000019CEE00, j9thread_t:0x00007F2EDEFDB380, java/lang/Thread:0x00000000E2E30C98, state:CW, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x47, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0xDF6D, native priority:0x5, native policy:UNKNOWN, vmstate:CW, vm thread flags:0x01000101)
3XMTHREADINFO2 (native stack address range from:0x00007F2EDDDDD000, to:0x00007F2EDDE1E000, size:0x41000)
3XMCPUTIME CPU usage total: 7.811379586 secs, current category="Application"
3XMTHREADBLOCK Waiting on: com/ibm/ws/tcpchannel/internal/SimpleSync#0x00000000E30B31E8 Owned by: <unowned>
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at java/lang/Object.wait(Native Method)
4XESTACKTRACE at java/lang/Object.wait(Object.java:172(Compiled Code))
4XESTACKTRACE at com/ibm/ws/tcpchannel/internal/SimpleSync.simpleWait(SimpleSync.java:35(Compiled Code))
5XESTACKTRACE (entered lock: com/ibm/ws/tcpchannel/internal/SimpleSync#0x00000000E30B31E8, entry count: 1)
4XESTACKTRACE at com/ibm/ws/tcpchannel/internal/NioTCPWriteRequestContextImpl.processSyncWriteRequest(NioTCPWriteRequestContextImpl.java:77(Compiled Code))
4XESTACKTRACE at com/ibm/ws/tcpchannel/internal/TCPWriteRequestContextImpl.write(TCPWriteRequestContextImpl.java:119(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/HttpServiceContextImpl.synchWrite(HttpServiceContextImpl.java:2438(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/HttpServiceContextImpl.sendOutgoing(HttpServiceContextImpl.java:2174(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/inbound/HttpInboundServiceContextImpl.sendResponseBody(HttpInboundServiceContextImpl.java:753(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/outbound/HttpOutputStreamImpl.flushBuffers(HttpOutputStreamImpl.java:481(Compiled Code))
4XESTACKTRACE at com/ibm/ws/http/channel/internal/outbound/HttpOutputStreamImpl.flush(HttpOutputStreamImpl.java:579(Compiled Code))
4XESTACKTRACE at com/ibm/wsspi/http/ee7/HttpOutputStreamEE7.flush(HttpOutputStreamEE7.java:293(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer/osgi/response/WCOutputStream.flush(WCOutputStream.java:234(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer31/osgi/response/WCOutputStream31.flush(WCOutputStream31.java:599(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer/srt/SRTServletResponse.flushBuffer(SRTServletResponse.java:705(Compiled Code))
4XESTACKTRACE at com/ibm/ws/webcontainer/srt/SRTServletResponse.flushBuffer(SRTServletResponse.java:670(Compiled Code))
4XESTACKTRACE at org/springframework/http/server/ServletServerHttpResponse.flush(ServletServerHttpResponse.java:96(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.writeFrameInternal(AbstractHttpSockJsSession.java:365(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractSockJsSession.writeFrame(AbstractSockJsSession.java:322(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/StreamingSockJsSession.flushCache(StreamingSockJsSession.java:86(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractHttpSockJsSession.sendMessageInternal(AbstractHttpSockJsSession.java:302(Compiled Code))
5XESTACKTRACE (entered lock: java/lang/Object#0x00000000E30B30F8, entry count: 1)
4XESTACKTRACE at org/springframework/web/socket/sockjs/transport/session/AbstractSockJsSession.sendMessage(AbstractSockJsSession.java:166(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/ConcurrentWebSocketSessionDecorator.tryFlushMessageBuffer(ConcurrentWebSocketSessionDecorator.java:132(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/handler/ConcurrentWebSocketSessionDecorator.sendMessage(ConcurrentWebSocketSessionDecorator.java:104(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.sendToClient(StompSubProtocolHandler.java:440(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/StompSubProtocolHandler.handleMessageToClient(StompSubProtocolHandler.java:427(Compiled Code))
4XESTACKTRACE at org/springframework/web/socket/messaging/SubProtocolWebSocketHandler.handleMessage(SubProtocolWebSocketHandler.java:338(Compiled Code))
WebSphere Application Server 16.0.0.3 (1.0.14.cl160320160831-1555) on IBM J9 VM, version pxa6480sr3fp11-20160817_02 (SR3 FP11) (en_US)
Spring-websocket 4.3.3.RELEASE
It looks like SRTServletResponse.flushBuffer implementation can lock threads infinitely,
SimpleSync.simpleWait never relese the thread. I tried to set tcpOptions inactivityTimeout, but it did't help.
WebSphere bug ?
We are having the same issue.
No workaround or fix yet.
Details:
Java 8.0.65
Windows Server 2012 R2 x64
Spring Boot 1.3.6.RELEASE
Spring Websocket 4.2.6.RELEASE
Undertow 1.3.22.Final
XNIO 3.3.6.Final
I have a problem where the receiver restarts again and again.
I am using Spark 1.6.1.
I use Spark Streaming to receive from a streaming, then use map to deserialize pb data.
My testing contains two cases:
Just receive the data and print directly: the app is stable
Receive and deserialize: this produces problems. The occurrence time is not regular.
There is 500Mb/min. I have set executor memory at 8GB. The problem is just like something allocate memory extremely. But, I don't know why.
My code:
val conf = new SparkConf().setAppName(args(8))
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
conf.set("spark.streaming.stopGracefullyOnShutdown", "true")
conf.set("spark.streaming.backpressure.enabled","true")
conf.set("spark.speculation","true")
val sc = new SparkContext(conf)
val ssc = new StreamingContext(sc, Seconds(args(7).toInt))
val bigPipeStreams = (1 to args(3).toInt).map{
i => ssc.networkStream(
new MyBigpipeLogagentReceiver(args(0),args(1),args(2),i,args(4),args(5),args(6).toInt)
)
}
val lines = ssc.union(bigPipeStreams)
def deserializePbData(value: String) : String = {
if (null == value || value.isEmpty) {
return ""
}
var cuid = ""
var os = ""
var channel = ""
var sv = ""
var resid = ""
var appid = ""
var prod = ""
try { //if exception,useless data,just drop it
val timeStrIndex = value.indexOf(",\"time_str\"")
var strAfterTruncation = ""
if (-1 != timeStrIndex) {
strAfterTruncation = value.substring(0,timeStrIndex) + "}"
} else {
strAfterTruncation = value
}
val jsonData = JSONObject.fromObject(strAfterTruncation)
//val jsonData = value.getAsJsonArray()
val binBody = jsonData.getString("bin_body")
val pbData = binBody.substring(1,binBody.length()-1).split(",").foldLeft(ArrayBuffer.empty[Byte])((b,a) => b +java.lang.Byte.parseByte(a)).drop(8).toArray
Lighttpd.lighttpd_log.parseFrom(pbData).getRequest().getUrl().getUrlFields().getAutokvList().asScala.foreach(a =>
a.getKey() match {
case "cuid" => cuid += a.getValue()
case "os" => os += a.getValue()
case "channel" => channel += a.getValue()
case "sv" => sv += a.getValue()
case "resid" => resid += a.getValue()
case "appid" => appid += a.getValue()
case "prod" => prod += a.getValue()
case _ => null
}
)
val decodeCuid = URLDecoder.decode(cuid, "UTF-8")
os = os.toLowerCase()
if (os.matches("android(.*)")) {
os = "android"
} else if (os.matches("iphone(.*)")) {
os = "iphone"
} else if (os.matches("ipad(.*)")) {
os = "ipad"
} else if (os.matches("s60(.*)")) {
os = "symbian"
} else if (os.matches("wp7(.*)")) {
os = "wp7"
} else if (os.matches("wp(.*)")) {
os = "wp"
} else if (os.matches("tizen(.*)")) {
os = "tizen"
val ifHasLogid = Lighttpd.lighttpd_log.parseFrom(pbData).hasLogid()
val time = Lighttpd.lighttpd_log.parseFrom(pbData).getTime()
if (ifHasLogid) {
val logid = Lighttpd.lighttpd_log.parseFrom(pbData).getLogid()
if (logid.isEmpty || logid.toString().equals("-") || !resid.toString().equals("01") || channel.isEmpty |!appid.isEmpty || !prod.isEmpty) {
""
} else {
decodeCuid + "\001" + os + "\001" + channel + "\001" + sv + "\001" + "1" + "\001" + "1" + "\001" + time + "\n"
}
} else {
""
}
} catch {
case _:Throwable => ""
}
}
lines.map(parseData).print()
The error text:
016-07-12T12:00:01.546+0800: 5096.643: [GC (Allocation Failure)
Desired survivor size 442499072 bytes, new threshold 1 (max 15)
[PSYoungGen: 0K->0K(2356736K)] 5059009K->5059009K(7949312K), 0.0103342 secs] [Times: user=0.21 sys=0.00, real=0.01 secs]
2016-07-12T12:00:01.556+0800: 5096.654: [Full GC (Allocation Failure) [PSYoungGen: 0K->0K(2356736K)] [ParOldGen: 5059009K->5057376K(5592576K)] 5059009K->5057376K(7949312K), [Metaspace: 44836K->44490K(1089536K)], 0.8769617 secs] [Times: user=17.88 sys=0.04, real=0.88 secs]
2016-07-12T12:00:02.434+0800: 5097.531: Total time for which application threads were stopped: 1.2951974 seconds, Stopping threads took: 0.0000662 seconds
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid24310.hprof ...
2016-07-12T12:00:30.960+0800: 5126.057: Total time for which application threads were stopped: 28.5260812 seconds, Stopping threads took: 0.0000995 seconds
Heap dump file created [5211252802 bytes in 28.526 secs]
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill %p"
# Executing /bin/sh -c "kill 24310"...
2016-07-12T12:00:31.589+0800: 5126.686: Total time for which application threads were stopped: 0.6289627 seconds, Stopping threads took: 0.0001258 seconds
2016-07-12T12:00:31.595+0800: 5126.692: Total time for which application threads were stopped: 0.0004822 seconds, Stopping threads took: 0.0001493 seconds
2016-07-12 12:00:31.597 [Thread-5] ERROR [Logging.scala:95] - Uncaught exception in thread Thread[Thread-5,5,main]
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236) ~[na:1.8.0_51]
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118) ~[na:1.8.0_51]
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) ~[na:1.8.0_51]
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) ~[na:1.8.0_51]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_51]
at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) ~[na:1.8.0_51]
at com.esotericsoftware.kryo.io.Output.flush(Output.java:155) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at com.esotericsoftware.kryo.io.Output.require(Output.java:135) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at com.esotericsoftware.kryo.io.Output.writeString_slow(Output.java:420) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at com.esotericsoftware.kryo.io.Output.writeString(Output.java:326) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at com.esotericsoftware.kryo.serializers.DefaultSerializers$StringSerializer.write(DefaultSerializers.java:153) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at com.esotericsoftware.kryo.serializers.DefaultSerializers$StringSerializer.write(DefaultSerializers.java:146) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:194) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:153) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.storage.BlockManager.dataSerializeStream(BlockManager.scala:1196) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.storage.BlockManager.dataSerialize(BlockManager.scala:1202) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:858) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:645) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.BlockManagerBasedBlockHandler.storeBlock(ReceivedBlockHandler.scala:77) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl.pushAndReportBlock(ReceiverSupervisorImpl.scala:157) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl.pushArrayBuffer(ReceiverSupervisorImpl.scala:128) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.ReceiverSupervisorImpl$$anon$3.onPushBlock(ReceiverSupervisorImpl.scala:109) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.BlockGenerator.pushBlock(BlockGenerator.scala:296) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.BlockGenerator.org$apache$spark$streaming$receiver$BlockGenerator$$keepPushingBlocks( BlockGenerator.scala:268) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.streaming.receiver.BlockGenerator$$anon$1.run(BlockGenerator.scala:109) ~[ spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
2016-07-12 12:00:31.600 [SIGTERM handler] ERROR [SignalLogger.scala:57] - RECEIVED SIGNAL 15: SIGTERM
2016-07-12T12:00:31.611+0800: 5126.708: Total time for which application threads were stopped: 0.0005602 seconds, Stopping threads took: 0.0001765 seconds
2016-07-12T12:00:31.617+0800: 5126.714: Total time for which application threads were stopped: 0.0004800 seconds, Stopping threads took: 0.0001412 seconds
2016-07-12 12:00:32.483 [Bigpipe Receiver-SendThread(cq01-bigpipe-proxy01.cq01.baidu.com:2181)] WARN [ClientCnxnSocket.java:139] - Connected to an old server; r-o mode will be unavailable
2016-07-12T12:00:32.507+0800: 5127.604: Total time for which application threads were stopped: 0.0004604 seconds, Stopping threads took: 0.0001198 seconds
2016-07-12T12:00:32.509+0800: 5127.606: Total time for which application threads were stopped: 0.0002919 seconds, Stopping threads took: 0.0001800 seconds
2016-07-12T12:00:32.509+0800: 5127.607: Total time for which application threads were stopped: 0.0002692 seconds, Stopping threads took: 0.0001612 seconds
2016-07-12 12:00:32.549 [Bigpipe Receiver-SendThread(tc-bigpipe-proxy03.tc.baidu.com:2181)] WARN [ClientCnxnSocket.java:139] - Connected to an old server; r-o mode will be unavailable
2016-07-12T12:00:34.220+0800: 5129.317: [GC (Allocation Failure)
Desired survivor size 424148992 bytes, new threshold 2 (max 15)
[PSYoungGen: 1931776K->188775K(2363904K)] 6989152K->5246152K(7956480K), 0.2569385 secs] [Times: user=0.00 sys=5.19, real=0.26 secs]
2016-07-12T12:00:34.477+0800: 5129.575: Total time for which application threads were stopped: 0.2575019 seconds, Stopping threads took: 0.0000384 seconds
2016-07-12T12:00:35.478+0800: 5130.575: Total time for which application threads were stopped: 0.0002786 seconds, Stopping threads took: 0.0000424 seconds
2016-07-12T12:00:37.600+0800: 5132.697: [GC (Allocation Failure)
Desired survivor size 482344960 bytes, new threshold 3 (max 15)
[PSYoungGen: 2120551K->387013K(2268160K)] 7177928K->5444389K(7860736K), 0.5153031 secs] [Times: user=0.00 sys=9.89, real=0.52 secs]
2016-07-12T12:00:38.116+0800: 5133.213: Total time for which application threads were stopped: 0.5157529 seconds, Stopping threads took: 0.0000427 seconds
2016-07-12T12:00:40.116+0800: 5135.213: Total time for which application threads were stopped: 0.0003171 seconds, Stopping threads took: 0.0001000 seconds
2016-07-12T12:00:40.419+0800: 5135.516: [GC (Allocation Failure)
Desired survivor size 599785472 bytes, new threshold 2 (max 15)
[PSYoungGen: 2240965K->471033K(2324992K)] 7298341K->5633517K(7917568K), 0.3621433 secs] [Times: user=0.12 sys=7.11, real=0.36 secs]
2016-07-12T12:00:40.781+0800: 5135.878: Total time for which application threads were stopped: 0.3626080 seconds, Stopping threads took: 0.0000429 seconds
2016-07-12T12:00:41.781+0800: 5136.879: Total time for which application threads were stopped: 0.0003301 seconds, Stopping threads took: 0.0000947 seconds
2016-07-12T12:00:43.108+0800: 5138.205: [GC (Allocation Failure)
Desired survivor size 620756992 bytes, new threshold 3 (max 15)
[PSYoungGen: 2324985K->378481K(2054656K)] 7487469K->5831048K(7647232K), 0.2593685 secs] [Times: user=0.66 sys=4.96, real=0.26 secs]
2016-07-12T12:00:43.368+0800: 5138.465: [Full GC (Ergonomics) [PSYoungGen: 378481K->0K(2054656K)] [ParOldGen: 5452566K->4713601K(5592576K)] 5831048K->4713601K(7647232K), [Metaspace: 44635K->44635K(1089536K)], 4.3137405 secs] [Times: user=9.78 sys=74.53, real=4.31 secs]
2016-07-12T12:00:47.682+0800: 5142.779: Total time for which application threads were stopped: 4.5736603 seconds, Stopping threads took: 0.0000449 seconds
2016-07-12T12:00:47.682+0800: 5142.779: Total time for which application threads were stopped: 0.0002430 seconds, Stopping threads took: 0.0000856 seconds
2016-07-12T12:00:49.954+0800: 5145.052: [GC (Allocation Failure)
Desired survivor size 597688320 bytes, new threshold 4 (max 15)
[PSYoungGen: 1583616K->161266K(2189824K)] 6297217K->4874867K(7782400K), 0.0388138 secs] [Times: user=0.00 sys=0.84, real=0.04 secs]
2016-07-12T12:00:49.993+0800: 5145.091: Total time for which application threads were stopped: 0.0392926 seconds, Stopping threads took: 0.0000449 seconds
2016-07-12T12:00:51.903+0800: 5147.000: [GC (Allocation Failure)
Desired survivor size 596115456 bytes, new threshold 5 (max 15)
[PSYoungGen: 1744882K->324587K(2213888K)] 6458483K->5038189K(7806464K), 0.0334029 secs] [Times: user=0.69 sys=0.03, real=0.04 secs]
2016-07-12T12:00:51.936+0800: 5147.034: Total time for which application threads were stopped: 0.0338707 seconds, Stopping threads took: 0.0000404 seconds
2016-07-12T12:00:53.942+0800: 5149.039: [GC (Allocation Failure)
Desired survivor size 654835712 bytes, new threshold 6 (max 15)
[PSYoungGen: 1954795K->490438K(2120704K)] 6668397K->5204039K(7713280K), 0.0441762 secs] [Times: user=0.95 sys=0.02, real=0.05 secs]
2016-07-12T12:00:53.986+0800: 5149.083: Total time for which application threads were stopped: 0.0446174 seconds, Stopping threads took: 0.0000456 seconds
2016-07-12T12:00:56.102+0800: 5151.199: [GC (Allocation Failure)
Desired survivor size 763887616 bytes, new threshold 5 (max 15)
[PSYoungGen: 2120646K->639467K(1943552K)] 6834247K->5370280K(7536128K), 0.1124828 secs] [Times: user=1.07 sys=1.30, real=0.11 secs]
2016-07-12T12:00:56.214+0800: 5151.312: Total time for which application threads were stopped: 0.1129348 seconds, Stopping threads took: 0.0000396 seconds
2016-07-12T12:00:57.784+0800: 5152.881: [GC (Allocation Failure)
Desired survivor size 895483904 bytes, new threshold 4 (max 15)
[PSYoungGen: 1943531K->745977K(2050048K)] 6674344K->5504073K(7642624K), 0.0971717 secs] [Times: user=1.20 sys=0.67, real=0.10 secs]
2016-07-12T12:00:57.881+0800: 5152.979: Total time for which application threads were stopped: 0.0977363 seconds, Stopping threads took: 0.0000941 seconds
2016-07-12T12:00:59.406+0800: 5154.504: [GC (Allocation Failure)
Desired survivor size 935329792 bytes, new threshold 5 (max 15)
[PSYoungGen: 2050041K->599188K(1715200K)] 6808137K->5647517K(7307776K), 0.3651465 secs] [Times: user=0.98 sys=5.88, real=0.37 secs]
2016-07-12T12:00:59.772+0800: 5154.869: Total time for which application threads were stopped: 0.3656089 seconds, Stopping threads took: 0.0000479 seconds
2016-07-12T12:01:00.968+0800: 5156.066: [GC (Allocation Failure)
Desired survivor size 954204160 bytes, new threshold 4 (max 15)
[PSYoungGen: 1568404K->697830K(1667072K)] 6616733K->5746159K(7259648K), 0.0978955 secs] [Times: user=1.91 sys=0.04, real=0.09 secs]
2016-07-12T12:01:01.066+0800: 5156.164: Total time for which application threads were stopped: 0.0983759 seconds, Stopping threads took: 0.0000482 seconds
2016-07-12T12:01:02.189+0800: 5157.287: [GC (Allocation Failure)
Desired survivor size 954204160 bytes, new threshold 3 (max 15)
[PSYoungGen: 1667046K->465454K(1864192K)] 6715375K->5855655K(7456768K), 0.1261993 secs] [Times: user=2.41 sys=0.29, real=0.12 secs]
2016-07-12T12:01:02.316+0800: 5157.413: [Full GC (Ergonomics) [PSYoungGen: 465454K->65236K(1864192K)] [ParOldGen: 5390200K->5592328K(5592576K)] 5855655K->5657564K(7456768K), [Metaspace: 44635K->44635K(1089536K)], 3.2729437 secs] [Times: user=12.34 sys=57.11, real=3.28 secs]
2016-07-12T12:01:05.589+0800: 5160.686: Total time for which application threads were stopped: 3.3998619 seconds, Stopping threads took: 0.0000521 seconds
2016-07-12T12:01:05.589+0800: 5160.686: Total time for which application threads were stopped: 0.0002330 seconds, Stopping threads took: 0.0000949 seconds
2016-07-12T12:01:05.688+0800: 5160.785: Total time for which application threads were stopped: 0.0002935 seconds, Stopping threads took: 0.0000514 seconds
Heap
PSYoungGen total 1864192K, used 146620K [0x0000000715580000, 0x00000007c0000000, 0x00000007c0000000)
eden space 932352K, 8% used [0x0000000715580000,0x000000071a4fa138,0x000000074e400000)
from space 931840K, 7% used [0x0000000787200000,0x000000078b1b5290,0x00000007c0000000)
to space 931840K, 0% used [0x000000074e400000,0x000000074e400000,0x0000000787200000)
ParOldGen total 5592576K, used 5592328K [0x00000005c0000000, 0x0000000715580000, 0x0000000715580000)
object space 5592576K, 99% used [0x00000005c0000000,0x00000007155420a8,0x0000000715580000)
Metaspace used 44654K, capacity 44990K, committed 45864K, reserved 1089536K
class space used 6212K, capacity 6324K, committed 6440K, reserved 1048576K
New Error: I think it's the upload error which call oom problem.I'd like to know how to fix this upload error?
2016-07-15 11:41:47.307 [shuffle-client-0] ERROR [TransportChannelHandler.java:128] - Connection to nmg01-taihang-d10207.nmg01.baidu.com/10.76.48.22:30456 has been quiet for 120000 ms while there are outstanding requests. Assuming connection is dead; please adjust spark.network.timeout if this is wrong.
2016-07-15 11:41:47.309 [shuffle-client-0] ERROR [TransportResponseHandler.java:122] - Still have 1 requests outstanding when connection from nmg01-taihang-d10207.nmg01.baidu.com/10.76.48.22:30456 is closed
2016-07-15 11:41:47.314 [shuffle-client-0] ERROR [Logging.scala:95] - Error while uploading block input-0-1468553896200
java.io.IOException: Connection from nmg01-taihang-d10207.nmg01.baidu.com/10.76.48.22:30456 closed
at org.apache.spark.network.client.TransportResponseHandler.channelUnregistered(TransportResponseHandler.java:124) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.network.server.TransportChannelHandler.channelUnregistered(TransportChannelHandler.java:94) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.ChannelInboundHandlerAdapter.channelUnregistered(ChannelInboundHandlerAdapter.java:53) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.ChannelInboundHandlerAdapter.channelUnregistered(ChannelInboundHandlerAdapter.java:53) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.ChannelInboundHandlerAdapter.channelUnregistered(ChannelInboundHandlerAdapter.java:53) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:739) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:659) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) [spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
2016-07-15T11:41:47.316+0800: 2176.487: Total time for which application threads were stopped: 0.0002632 seconds, Stopping threads took: 0.0000521 seconds
2016-07-15 11:41:47.316 [Thread-5] WARN [Logging.scala:91] - Failed to replicate input-0-1468553896200 to BlockManagerId(2, nmg01-taihang-d10207.nmg01.baidu.com, 30456), failure #0
java.io.IOException: Connection from nmg01-taihang-d10207.nmg01.baidu.com/10.76.48.22:30456 closed
at org.apache.spark.network.client.TransportResponseHandler.channelUnregistered(TransportResponseHandler.java:124) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at org.apache.spark.network.server.TransportChannelHandler.channelUnregistered(TransportChannelHandler.java:94) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.ChannelInboundHandlerAdapter.channelUnregistered(ChannelInboundHandlerAdapter.java:53) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.ChannelInboundHandlerAdapter.channelUnregistered(ChannelInboundHandlerAdapter.java:53) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.ChannelInboundHandlerAdapter.channelUnregistered(ChannelInboundHandlerAdapter.java:53) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelUnregistered(AbstractChannelHandlerContext.java:158) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelUnregistered(AbstractChannelHandlerContext.java:144) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.DefaultChannelPipeline.fireChannelUnregistered(DefaultChannelPipeline.java:739) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:659) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111) ~[spark-assembly-1.6.1.0-baidu-SNAPSHOT-hadoop2.5.1.3-baidu-SNAPSHOT.jar:1.6.1.0-baidu-SNAPSHOT]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_51]
2016-07-15T11:41:48.316+0800: 2177.487: Total time for which application threads were stopped: 0.0003391 seconds, Stopping threads took: 0.0000979 seconds
2016-07-15T11:41:51.312+0800: 2180.483: [GC (Allocation Failure) --[PSYoungGen: 2894863K->2894863K(3007488K)] 8299519K->9550273K(9998336K), 0.7462118 secs] [Times: user=9.78 sys=0.02, real=0.74 secs]
2016-07-15T11:41:52.059+0800: 2181.230: [Full GC (Ergonomics) [PSYoungGen: 2894863K->0K(3007488K)] [ParOldGen: 6655410K->6895736K(6990848K)] 9550273K->6895736K(9998336K), [Metaspace: 44409K->44409K(1087488K)], 0.4061892 secs] [Times: user=7.50 sys=0.01, real=0.41 secs]
Your code appears to have an error in structure. In the process of looking at your code (to re-indenting it to reflect the structure as posted), I found that your last else if statement:
} else if (os.matches("tizen(.*)")) {
os = "tizen"
opens a block, but does not close the block where it "should". Instead, the block is actually terminated with:
} catch {
The full code as it appears was intended (and re-indented) is:
val conf = new SparkConf().setAppName(args(8))
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
conf.set("spark.streaming.stopGracefullyOnShutdown", "true")
conf.set("spark.streaming.backpressure.enabled","true")
conf.set("spark.speculation","true")
val sc = new SparkContext(conf)
val ssc = new StreamingContext(sc, Seconds(args(7).toInt))
val bigPipeStreams = (1 to args(3).toInt).map{
i => ssc.networkStream(
new MyBigpipeLogagentReceiver(args(0),args(1),args(2),i,args(4),args(5),args(6).toInt)
)
}
val lines = ssc.union(bigPipeStreams)
def deserializePbData(value: String) : String = {
if (null == value || value.isEmpty) {
return ""
}
var cuid = ""
var os = ""
var channel = ""
var sv = ""
var resid = ""
var appid = ""
var prod = ""
try { //if exception,useless data,just drop it
val timeStrIndex = value.indexOf(",\"time_str\"")
var strAfterTruncation = ""
if (-1 != timeStrIndex) {
strAfterTruncation = value.substring(0,timeStrIndex) + "}"
} else {
strAfterTruncation = value
}
val jsonData = JSONObject.fromObject(strAfterTruncation)
//val jsonData = value.getAsJsonArray()
val binBody = jsonData.getString("bin_body")
val pbData = binBody.substring(1,binBody.length()-1).split(",").foldLeft(ArrayBuffer.empty[Byte])((b,a) => b +java.lang.Byte.parseByte(a)).drop(8).toArray
Lighttpd.lighttpd_log.parseFrom(pbData).getRequest().getUrl().getUrlFields().getAutokvList().asScala.foreach(a =>
a.getKey() match {
case "cuid" => cuid += a.getValue()
case "os" => os += a.getValue()
case "channel" => channel += a.getValue()
case "sv" => sv += a.getValue()
case "resid" => resid += a.getValue()
case "appid" => appid += a.getValue()
case "prod" => prod += a.getValue()
case _ => null
}
)
val decodeCuid = URLDecoder.decode(cuid, "UTF-8")
os = os.toLowerCase()
if (os.matches("android(.*)")) {
os = "android"
} else if (os.matches("iphone(.*)")) {
os = "iphone"
} else if (os.matches("ipad(.*)")) {
os = "ipad"
} else if (os.matches("s60(.*)")) {
os = "symbian"
} else if (os.matches("wp7(.*)")) {
os = "wp7"
} else if (os.matches("wp(.*)")) {
os = "wp"
} else if (os.matches("tizen(.*)")) {
os = "tizen"
}
val ifHasLogid = Lighttpd.lighttpd_log.parseFrom(pbData).hasLogid()
val time = Lighttpd.lighttpd_log.parseFrom(pbData).getTime()
if (ifHasLogid) {
val logid = Lighttpd.lighttpd_log.parseFrom(pbData).getLogid()
if (logid.isEmpty || logid.toString().equals("-") || !resid.toString().equals("01") || channel.isEmpty |!appid.isEmpty || !prod.isEmpty) {
""
} else {
decodeCuid + "\001" + os + "\001" + channel + "\001" + sv + "\001" + "1" + "\001" + "1" + "\001" + time + "\n"
}
} else {
""
}
} catch {
case _:Throwable => ""
}
}
lines.map(parseData).print()
I have not checked your code for functionality. This is just a syntax/structure issue that stood out when very briefly looking at your the code you posted.
It was working fine until my PC got restarted due to power outage and now the publishing gives the following error.. Please help me solve this
The publish encountered some problems and the application may not have been installed or it may have been successfully installed but was unable to start. View the Details to display the exceptions and server logs that occurred during the publish.
======== SPMEAR ========
Application Failed to Install. SPMEAR
Application Failed to Install. SPMEAR
Please see server logs for more details.
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R java.lang.NullPointerException
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.bla.model.CompositionUnitConfig.readJ2EEAppDeployment(CompositionUnitConfig.java:291)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.bla.model.CompositionUnitConfig.read(CompositionUnitConfig.java:225)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.bla.model.CompositionUnitFactoryImpl.readCompositionUnitFromCompositionUnitSpec(CompositionUnitFactoryImpl.java:76)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.application.task.ValidateBLATask._validateBLAUninstall(ValidateBLATask.java:548)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.application.task.ValidateBLATask._performTaskAsSystem(ValidateBLATask.java:122)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.application.task.ValidateBLATask$1.run(ValidateBLATask.java:91)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:4189)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.security.auth.ContextManagerImpl.runAsSpecified(ContextManagerImpl.java:4347)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.application.task.ValidateBLATask.performTask(ValidateBLATask.java:88)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at com.ibm.ws.management.application.SchedulerImpl.run(SchedulerImpl.java:284)
[8/2/13 23:17:58:057 IST] 0000002d SystemErr R at java.lang.Thread.run(Thread.java:735)
This is the stack trace
java.lang.NullPointerException
at com.ibm.etools.wrd.websphere.v7.internal.operations.ApplicationUpdateOperation.execute(Unknown Source)
at com.ibm.etools.wrd.websphere.v7.internal.operations.WASPublisherImpl.publishDelta(Unknown Source)
at com.ibm.ws.ast.st.v7.core.internal.WASServerBehaviour.publishWrdDelta(Unknown Source)
at com.ibm.ws.ast.st.v7.core.internal.WASServerBehaviour.publishModules(Unknown Source)
at org.eclipse.wst.server.core.model.ServerBehaviourDelegate.publish(Unknown Source)
at org.eclipse.wst.server.core.model.ServerBehaviourDelegate.publish(Unknown Source)
at org.eclipse.wst.server.core.internal.Server.publishImpl(Unknown Source)
at org.eclipse.wst.server.core.internal.Server$PublishJob.run(Unknown Source)
at org.eclipse.core.internal.jobs.Worker.run(Unknown Source)
maybe a corrupted files inside your tmp ... try to clean you /tmp and restart if not solved .. try to change your context root as a temp solution
In order to fix this :
Go to Window -> Preference->Server->websphere Application Server.
enable Allow applications containing errors to be published on a server option.
BR,
Priya
I am trying to Install Oracle 11G on Windows Xp 64 bit. The installer shows the progress bar for checking prerequisites and then crashes.
The log file is as below.
What can I do to solve the issue?
#
An unexpected error has been detected by HotSpot Virtual Machine:
#
EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000077d64e95,
pid=3580, tid=2860
#
Java VM: Java HotSpot(TM) 64-Bit Server VM (1.5.0_17-b04 mixed mode)
Problematic frame:
C [kernel32.dll+0x24e95]
#
--------------- T H R E A D ---------------
Current thread (0x0000000001ba0740):
JavaThread "AWT-EventQueue-0"
[_thread_in_native, id=2860]
siginfo: ExceptionCode=0xc0000005,
reading address 0x000000007f366f48
Registers: EAX=0x0000000000000000,
EBX=0x000000007f366f48,
ECX=0xffffffffffffffff,
EDX=0x0000000000000104
ESP=0x000000000776de70,
EBP=0x0000000000000000,
ESI=0x00000000000004e4,
EDI=0x000000007f366f48
EIP=0x0000000077d64e95,
EFLAGS=0x0000000000010246
Top of Stack: (sp=0x000000000776de70)
0x000000000776de70: 0000000000000006
0000000000000000 0x000000000776de80:
0000000000000000 0000000000000020
0x000000000776de90: 0000000000000005
00000000001956b0 0x000000000776dea0:
0000000010020200 00000000080b95e6
0x000000000776deb0: 0000000020a9c058
0000000077efb1e9 0x000000000776dec0:
0000000000000000 0000000077c4ef3d
0x000000000776ded0: 0000ebe94f6aad20
0000000001ba0740 0x000000000776dee0:
000000000776e388 000000001257ad50
0x000000000776def0: 0000000001e9cba1
0000000020a9c340 0x000000000776df00:
000000000000006d 000000000776e348
0x000000000776df10: 000000001257ad50
0000000006cbd5f9 0x000000000776df20:
000000000776dfe8 000000000776e0a8
0x000000000776df30: 0000000000000000
000007ff7f1e4273 0x000000000776df40:
000000000776e080 0000000000000104
0x000000000776df50: 0000ebe94f6aad20
00000000001956b0 0x000000000776df60:
000000000016a090 000000000776dfe8
Instructions: (pc=0x0000000077d64e95)
0x0000000077d64e85: ff 33 c0 48 b9
ff ff ff ff ff ff ff ff 48 8b fb
0x0000000077d64e95: f2 ae 48 f7 d1
44 8b e9 e9 1e 86 00 00 90 90 90
Stack:
[0x0000000007670000,0x0000000007770000),
sp=0x000000000776de70, free
space=1015k Native frames: (J=compiled
Java code, j=interpreted, Vv=VM code,
C=native code) C
[kernel32.dll+0x24e95]
[error occurred during error
reporting, step 120, id 0xc0000005]
Java frames: (J=compiled Java code,
j=interpreted, Vv=VM code) j
sun.awt.shell.Win32ShellFolder2.getLinkLocation(JJZ)J+0
j
sun.awt.shell.Win32ShellFolder2.getLinkLocation(Z)Lsun/awt/shell/ShellFolder;+20
j
sun.awt.shell.Win32ShellFolder2.isDirectory()Z+69
j
sun.awt.shell.Win32ShellFolder2.isDirectory()Z+79
j
sun.awt.shell.Win32ShellFolderManager2.get(Ljava/lang/String;)Ljava/lang/Object;+156
j
sun.awt.shell.ShellFolder.get(Ljava/lang/String;)Ljava/lang/Object;+4
j
oracle.bali.ewt.olaf2.OracleFileChooserUI.installComponents(Ljavax/swing/JFileChooser;)V+794
j
javax.swing.plaf.basic.BasicFileChooserUI.installUI(Ljavax/swing/JComponent;)V+47
j
oracle.bali.ewt.olaf2.OracleFileChooserUI.installUI(Ljavax/swing/JComponent;)V+2
j
javax.swing.JComponent.setUI(Ljavax/swing/plaf/ComponentUI;)V+42
j
javax.swing.JFileChooser.updateUI()V+40
j
javax.swing.JFileChooser.setup(Ljavax/swing/filechooser/FileSystemView;)V+73
j
javax.swing.JFileChooser.(Ljava/io/File;Ljavax/swing/filechooser/FileSystemView;)V+133
j
javax.swing.JFileChooser.()V+9 j
oracle.install.commons.swing.XmlBeanStoreFileChooser.()V+1
j
oracle.install.commons.base.summary.SummaryPane.buildUI()V+25
j
oracle.install.commons.base.summary.SummaryPane.()V+35
j
oracle.install.commons.base.interview.common.view.SummaryGUI.()V+9
v ~StubRoutines::call_stub j
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Ljava/lang/reflect/Constructor;[Ljava/lang/Object;)Ljava/lang/Object;+0
j
sun.reflect.NativeConstructorAccessorImpl.newInstance([Ljava/lang/Object;)Ljava/lang/Object;+72
j
sun.reflect.DelegatingConstructorAccessorImpl.newInstance([Ljava/lang/Object;)Ljava/lang/Object;+5
j
java.lang.reflect.Constructor.newInstance([Ljava/lang/Object;)Ljava/lang/Object;+92
j
java.lang.Class.newInstance0()Ljava/lang/Object;+118
j
java.lang.Class.newInstance()Ljava/lang/Object;+15
j
oracle.install.commons.flow.ViewManager.getView(Ljava/lang/String;)Loracle/install/commons/flow/View;+72
j
oracle.install.commons.flow.State.getView()Loracle/install/commons/flow/View;+19
j
oracle.install.commons.flow.AbstractFlowExecutor.loadView(Loracle/install/commons/util/Graph;)V+20
j
oracle.install.commons.flow.AbstractGraphicalFlowExecutor.access$001(Loracle/install/commons/flow/AbstractGraphicalFlowExecutor;Loracle/install/commons/util/Graph;)V+2
j
oracle.install.commons.flow.AbstractGraphicalFlowExecutor$1.call()Ljava/lang/Void;+8
j
oracle.install.commons.flow.AbstractGraphicalFlowExecutor$1.call()Ljava/lang/Object;+1
j
java.util.concurrent.FutureTask$Sync.innerRun()V+22
j
java.util.concurrent.FutureTask.run()V+4
j
java.awt.event.InvocationEvent.dispatch()V+47
j
java.awt.EventQueue.dispatchEvent(Ljava/awt/AWTEvent;)V+26
j
java.awt.EventDispatchThread.pumpOneEventForHierarchy(ILjava/awt/Component;)Z+233
j
java.awt.EventDispatchThread.pumpEventsForHierarchy(ILjava/awt/Conditional;Ljava/awt/Component;)V+26
j
java.awt.EventDispatchThread.pumpEvents(ILjava/awt/Conditional;)V+4
j
java.awt.EventDispatchThread.pumpEvents(Ljava/awt/Conditional;)V+3
j
java.awt.EventDispatchThread.run()V+9
v ~StubRoutines::call_stub
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x0000000001ba0490 JavaThread "Image
Fetcher 0" daemon [_thread_blocked,
id=3088] 0x0000000001ba1cc0
JavaThread "TimerQueue" daemon
[_thread_blocked, id=3252]
0x0000000001ba1a10 JavaThread
"pool-1-thread-1" [_thread_blocked,
id=4912] 0x0000000001b9e700
JavaThread "DestroyJavaVM"
[_thread_blocked, id=3504]
0x0000000001ba1760 JavaThread "Worker
3" daemon [_thread_blocked, id=3660]
0x0000000001ba14b0 JavaThread "Worker
2" daemon [_thread_blocked, id=3720]
0x0000000001ba1200 JavaThread "Worker
1" daemon [_thread_blocked, id=1936]
0x0000000001ba0f50 JavaThread "Worker
0" daemon [_thread_blocked, id=4016]
=>0x0000000001ba0740 JavaThread "AWT-EventQueue-0" [_thread_in_native,
id=2860] 0x0000000001b9f720
JavaThread "AWT-Shutdown"
[_thread_blocked, id=2276]
0x0000000001b9ff30 JavaThread "Java2D
Disposer" daemon [_thread_blocked,
id=4064] 0x0000000001b9f9d0
JavaThread "AWT-Windows" daemon
[_thread_in_native, id=2376]
0x0000000001b9f1c0 JavaThread "Low
Memory Detector" daemon
[_thread_blocked, id=1764]
0x000000000557f720 JavaThread
"CompilerThread1" daemon
[_thread_blocked, id=4204]
0x000000000557f450 JavaThread
"CompilerThread0" daemon
[_thread_blocked, id=3604]
0x000000000557f180 JavaThread
"AdapterThread" daemon
[_thread_blocked, id=1976]
0x0000000001b9ef10 JavaThread "Signal
Dispatcher" daemon [_thread_blocked,
id=2296] 0x0000000001b9ec60
JavaThread "Finalizer" daemon
[_thread_blocked, id=3608]
0x0000000001b9e9b0 JavaThread
"Reference Handler" daemon
[_thread_blocked, id=2012]
Other Threads: 0x0000000001b8d980
VMThread [id=4856]
0x0000000001b8daa0 WatcherThread
[id=1224]
VM state:not at safepoint (normal
execution)
VM Mutex/Monitor currently owned by a
thread: None
Heap PSYoungGen total 62592K,
used 56702K [0x000000001d410000,
0x0000000021410000,
0x0000000021410000) eden space
59648K, 93% used
[0x000000001d410000,0x0000000020aab7d8,0x0000000020e50000)
from space 2944K, 26% used
[0x0000000020e50000,0x0000000020f140c0,0x0000000021130000)
to space 2880K, 0% used
[0x0000000021140000,0x0000000021140000,0x0000000021410000)
PSOldGen total 12800K, used
9744K [0x0000000015410000,
0x0000000016090000,
0x000000001d410000) object space
12800K, 76% used
[0x0000000015410000,0x0000000015d94300,0x0000000016090000)
PSPermGen total 54016K, used
38334K [0x0000000010010000,
0x00000000134d0000,
0x0000000015410000) object space
54016K, 70% used
[0x0000000010010000,0x000000001257f820,0x00000000134d0000)
Dynamic libraries: 0x0000000000400000
- 0x0000000000412000 C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\javaw.exe
0x0000000077ec0000 -
0x0000000077ff9000
C:\WINDOWS\system32\ntdll.dll
0x0000000077d40000 -
0x0000000077eb3000
C:\WINDOWS\system32\kernel32.dll
0x000007ff7fee0000 -
0x000007ff7ffe5000
C:\WINDOWS\system32\ADVAPI32.dll
0x000007ff7fd30000 -
0x000007ff7fec9000
C:\WINDOWS\system32\RPCRT4.dll
0x000007ff7e9c0000 -
0x000007ff7e9e2000
C:\WINDOWS\system32\Secur32.dll
0x0000000077c20000 -
0x0000000077d2c000
C:\WINDOWS\system32\USER32.dll
0x000007ff7fc90000 -
0x000007ff7fd2b000
C:\WINDOWS\system32\GDI32.dll
0x000007ff7fc00000 -
0x000007ff7fc86000
C:\WINDOWS\system32\msvcrt.dll
0x0000000180000000 -
0x00000001800f3000
c:\progra~1\quickh~1\quickh~2\wl_hoo~1.dll
0x000007ff7f190000 -
0x000007ff7fb9a000
C:\WINDOWS\system32\SHELL32.dll
0x000007ff7ef60000 -
0x000007ff7effb000
C:\WINDOWS\system32\SHLWAPI.dll
0x000007ff7f000000 -
0x000007ff7f187000
C:\WINDOWS\WinSxS\amd64_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.3790.3959_x-ww_0A7B2435\comctl32.dll 0x0000000008000000 -
0x0000000008561000
C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\server\jvm.dll 0x000007ff7e240000 -
0x000007ff7e290000
C:\WINDOWS\system32\WINMM.dll
0x0000000010000000 -
0x000000001000b000
C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\hpi.dll
0x000007ff7e380000 -
0x000007ff7e390000
C:\WINDOWS\system32\PSAPI.DLL
0x00000000003d0000 -
0x00000000003e1000
C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\verify.dll
0x0000000001e40000 -
0x0000000001e6c000
C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\java.dll 0x0000000001e70000 -
0x0000000001e86000
C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\zip.dll
0x0000000006b50000 -
0x0000000006d6b000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\awt.dll
0x000007ff78f10000 -
0x000007ff78f51000
C:\WINDOWS\system32\WINSPOOL.DRV
0x000007ff7d500000 -
0x000007ff7d539000
C:\WINDOWS\system32\IMM32.dll
0x000007ff57140000 -
0x000007ff573c5000
C:\WINDOWS\system32\ole32.dll
0x000007ff77230000 -
0x000007ff772af000
C:\WINDOWS\system32\uxtheme.dll
0x000007ff79190000 -
0x000007ff79202000
C:\WINDOWS\system32\ddraw.dll
0x000007ff793e0000 -
0x000007ff793e8000
C:\WINDOWS\system32\DCIMAN32.dll
0x000007ff7a410000 -
0x000007ff7a4af000
C:\WINDOWS\system32\MSCTF.dll
0x0000000007430000 -
0x00000000074aa000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\fontmanager.dll
0x00000000075b0000 -
0x00000000075cb000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\net.dll
0x000007ff77310000 -
0x000007ff77340000
C:\WINDOWS\system32\WS2_32.dll
0x000007ff77140000 -
0x000007ff7714c000
C:\WINDOWS\system32\WS2HELP.dll
0x0000000007650000 -
0x000000000765c000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\nio.dll
0x0000000007770000 -
0x00000000077ad000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\oui\lib\win64\oraInstaller.dll
0x000007ff7fbf0000 -
0x000007ff7fbfb000
C:\WINDOWS\system32\VERSION.dll
0x0000000078000000 -
0x0000000078060000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\oui\lib\win64\msvcrt.dll
0x000007ff771b0000 -
0x000007ff7722d000
C:\WINDOWS\System32\mswsock.dll
0x000007ff7e8f0000 -
0x000007ff7e93e000
C:\WINDOWS\system32\DNSAPI.dll
0x000007ff7e9f0000 -
0x000007ff7e9fb000
C:\WINDOWS\System32\winrnr.dll
0x000007ff7e950000 -
0x000007ff7e9b6000
C:\WINDOWS\system32\WLDAP32.dll
0x000007ff7ea00000 -
0x000007ff7ea07000
C:\WINDOWS\system32\rasadhlp.dll
0x000007ff57040000 -
0x000007ff57071000
C:\WINDOWS\system32\iphlpapi.dll
0x000007ff7e560000 -
0x000007ff7e58b000
C:\WINDOWS\system32\MPRAPI.dll
0x000007ff7e780000 -
0x000007ff7e7db000
C:\WINDOWS\system32\ACTIVEDS.dll
0x000007ff7e730000 -
0x000007ff7e770000
C:\WINDOWS\system32\adsldpc.dll
0x000007ff77370000 -
0x000007ff77409000
C:\WINDOWS\system32\NETAPI32.dll
0x000007ff7e3a0000 -
0x000007ff7e3d9000
C:\WINDOWS\system32\credui.dll
0x000007ff7e040000 -
0x000007ff7e05c000
C:\WINDOWS\system32\ATL.DLL
0x000007ff7ebc0000 -
0x000007ff7ecd6000
C:\WINDOWS\system32\OLEAUT32.dll
0x000007ff7e610000 -
0x000007ff7e625000
C:\WINDOWS\system32\rtutils.dll
0x000007ff77150000 -
0x000007ff77166000
C:\WINDOWS\system32\SAMLIB.dll
0x000007ff7dac0000 -
0x000007ff7dc38000
C:\WINDOWS\system32\SETUPAPI.dll
0x000007ff6d200000 -
0x000007ff6d293000
C:\WINDOWS\system32\hnetcfg.dll
0x000007ff77170000 -
0x000007ff7717b000
C:\WINDOWS\System32\wshtcpip.dll
0x0000000007970000 -
0x0000000007977000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\oui\lib\win64\orauts.dll
0x0000000007b20000 -
0x0000000007b2d000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\oui\lib\win64\orawsec11.dll
0x0000000078130000 -
0x00000000781f9000
C:\WINDOWS\WinSxS\amd64_Microsoft.VC80.CRT_1fc8b3b9a1e18e3b_8.0.50727.4027_x-ww_189D5EB0\MSVCR80.dll 0x0000000007c50000 -
0x0000000007cf4000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\oui\lib\win64\orasrvm11.dll
0x0000000007a90000 -
0x0000000007abd000 C:\Documents and
Settings\rrscw\Local
Settings\Temp\OraInstall2011-03-30_10-02-25AM\jdk\jre\bin\dcpr.dll 0x000007ff76200000 -
0x000007ff762ef000
C:\WINDOWS\WinSxS\amd64_Microsoft.Windows.Common-Controls_6595b64144ccf1df_5.82.3790.3959_x-ww_AB06DEB0\comctl32.dll 0x000007ff772d0000 -
0x000007ff772f1000
C:\WINDOWS\system32\MPR.dll
0x000007ff7ccc0000 -
0x000007ff7ccca000
C:\WINDOWS\System32\drprov.dll
0x000007ff649f0000 -
0x000007ff64a06000
C:\WINDOWS\System32\ntlanman.dll
0x000007ff65140000 -
0x000007ff65164000
C:\WINDOWS\System32\NETUI0.dll
0x000007ff650e0000 -
0x000007ff65138000
C:\WINDOWS\System32\NETUI1.dll
0x000007ff7ceb0000 -
0x000007ff7cebd000
C:\WINDOWS\System32\davclnt.dll
0x000007ff7ce50000 -
0x000007ff7ce91000
C:\WINDOWS\system32\apphelp.dll
0x000007ff7ea10000 -
0x000007ff7eaee000
C:\WINDOWS\system32\CLBCatQ.DLL
0x000007ff7eaf0000 -
0x000007ff7ebb6000
C:\WINDOWS\system32\COMRes.dll
0x000007ff7df70000 -
0x000007ff7df7c000
C:\WINDOWS\system32\LINKINFO.dll
0x000007ff7df80000 -
0x000007ff7dfb1000
C:\WINDOWS\system32\ntshrui.dll
VM Arguments: jvm_args:
-Doracle.installer.library_loc=C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\oui\lib\win64
-Doracle.installer.oui_loc=C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\oui
-Doracle.installer.bootstrap=TRUE -Doracle.installer.startup_location=I:\Setups\Oracle\database\install
-Doracle.installer.jre_loc=C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM\jdk/jre
-Doracle.installer.exec_location=I:\Setups\Oracle\database\install\oui.exe
-Doracle.installer.nlsEnabled=TRUE -Doracle.installer.prereqConfigLoc= -Xmx192m java_command: oracle.install.ivw.db.driver.DBInstaller
-scratchPath C:\DOCUME~1\rrscw\LOCALS~1\Temp\OraInstall2011-03-30_10-02-25AM
-sourceLoc I:\Setups\Oracle\database\install../stage/products.xml
-sourceType network -timestamp 2011-03-30_10-02-25AM Launcher Type:
SUN_STANDARD
Environment Variables:
PATH=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
Files (x86)\Common Files\Roxio
Shared\DLLShared\;C:\Program Files
(x86)\Common Files\Roxio
Shared\10.0\DLLShared\;C:\WINDOWS\system32\WindowsPowerShell\v1.0;C:\Program
Files (x86)\Microsoft SQL
Server\100\Tools\Binn\;C:\Program
Files (x86)\Microsoft SQL
Server\100\DTS\Binn\;C:\Program
Files\R\R-2.12.2\library\rscproxy\libs\i386;C:\Program
Files\R\R-2.12.2\bin\i386;C:\Program
Files (x86)\Microsoft SQL
Server\100\Tools\Binn\VSShell\Common7\IDE\
USERNAME=rrscw OS=Windows_NT
PROCESSOR_IDENTIFIER=EM64T Family 6
Model 26 Stepping 5, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows XP 64bit Build 3790
Service Pack 2
CPU:total 4 em64t ht
Memory: 4k page, physical
6273948k(4849464k free), swap
7972228k(7161264k free)
vm_info: Java HotSpot(TM) 64-Bit
Server VM (1.5.0_17-b04) for
windows-amd64, built on Nov 10 2008
01:04:27 by "java_re" with unknown MS
VC++:1400
I had the same issue and after some searching found this solution which has you remove all shortcuts from your Windows Desktop. Apparently the installer chokes on these for some reason. Worked like a charm.
Oracle sometimes has problems installing from a directory with a space in it.