I'm debugging a Cocoa app, which calls the system libraries. I get debug symbols for my own code, but not the system libraries, which stops me debugging any further. How can I get debug symbols for the system library.
I'm using gdb, compiling against the 10.5 SDK on 10.6.4 (gcc 4.2).
Here is the stack-trace I'd like to get symbols for:
(gdb) bt
#0 0x93713e43 in CFQSortArray ()
#1 0x936f4c49 in CFArraySortValues ()
#2 0x958f0bc0 in ColorSyncProfileCopyTagSignatures ()
#3 0x9591d218 in CMMProfile::Usable ()
#4 0x9591cb5d in DoValidateProfile ()
#5 0x9591cc75 in AppleCMMValidateProfile ()
#6 0x958f1e52 in ColorSyncProfileVerify ()
#7 0x91b13b88 in validateProfile ()
#8 0x91b13aed in CMSValidateProfile ()
#9 0x93b27f6c in CGCMSInterfaceValidateProfile ()
#10 0x93b27f4c in CGCMSUtilsValidateProfile ()
#11 0x93b27e4b in CGColorSpaceCreateICCBased ()
#12 0x93b27203 in create_generic_color_space ()
#13 0x93b26f92 in CGColorSpaceCreateWithIndex ()
#14 0x90228ad5 in +[NSColorSpace specialColorSpaceWithID:] ()
#15 0x90228a0c in +[NSColorSpace genericRGBColorSpace] ()
#16 0x9022881e in -[NSBitmapImageRep _bitmapImageRep_setColorSpaceName:] ()
#17 0x904f6e39 in -[NSBitmapImageRep initWithBitmapDataPlanes:pixelsWide:pixelsHigh:bitsPerSample:samplesPerPixel:hasAlpha:isPlanar:colorSpaceName:bitmapFormat:bytesPerRow:bitsPerPixel:] ()
#18 0x902282ae in -[NSBitmapImageRep _initWithSharedBitmap:rect:] ()
#19 0x90227efe in -[NSImage _addRepresentationWithSharedKitWindow:rect:] ()
#20 0x90226d06 in +[NSImage _findSystemImageNamed:] ()
#21 0x90226a48 in +[NSImage imageNamed:] ()
#22 0x902268ea in -[NSCustomResource _loadImageWithName:] ()
#23 0x9022681e in -[NSCustomResource loadImageWithName:] ()
#24 0x90225d5f in -[NSCustomResource awakeAfterUsingCoder:] ()
#25 0x915a2208 in _decodeObjectBinary ()
#26 0x915a14e4 in _decodeObject ()
#27 0x9022556f in -[NSMenuItem initWithCoder:] ()
#28 0x915a21ec in _decodeObjectBinary ()
#29 0x915a2ff8 in -[NSKeyedUnarchiver _decodeArrayOfObjectsForKey:] ()
#30 0x915a3665 in -[NSArray(NSArray) initWithCoder:] ()
#31 0x915a21ec in _decodeObjectBinary ()
#32 0x915a14e4 in _decodeObject ()
#33 0x90229ff0 in -[NSMenu initWithCoder:] ()
#34 0x915a21ec in _decodeObjectBinary ()
#35 0x915a2ff8 in -[NSKeyedUnarchiver _decodeArrayOfObjectsForKey:] ()
#36 0x915a3665 in -[NSArray(NSArray) initWithCoder:] ()
#37 0x915a21ec in _decodeObjectBinary ()
#38 0x915a14e4 in _decodeObject ()
#39 0x9021ff29 in -[NSIBObjectData initWithCoder:] ()
#40 0x915a21ec in _decodeObjectBinary ()
#41 0x915a14e4 in _decodeObject ()
#42 0x9021f4a8 in loadNib ()
#43 0x9021eb5b in +[NSBundle(NSNibLoading) _loadNibFile:nameTable:withZone:ownerBundle:] ()
#44 0x9021e811 in +[NSBundle(NSNibLoading) loadNibFile:externalNameTable:withZone:] ()
As described in this apple TechNote, you want to set DYLD_IMAGE_SUFFIX to _debug.
http://developer.apple.com/library/mac/#technotes/tn2004/tn2124.html
there are further problems then (not having the right version on the right system) which are hard to overcome, so I haven't tested this.
I doubt you'd find anything but link symbol names in system libraries. Certainly no line number info, and no source code (it's a closed-source commercial product). The best you can do is step through the assembler instructions in these functions.
You can use gdb finish command to run until selected stack frame returns. Do that several times to return to your application code. Or just next your way over library function calls.
Related
I've been tracking down an occasional crash in my app, which comes from an EXC_BAD_ACCESS KERN_PROTECTION_FAILURE, and I finally found something that it could be by using the Address Sanitizer. Unfortunately, I have no idea what the ASan output is telling me.
I've set up my Core Data stack based on this tutorial's stack. The ASan warning warning shows up here:
func mergeChanges(from transactions: [NSPersistentHistoryTransaction]) {
let context = viewContext
context.perform {
for transaction in transactions {
let note = transaction.objectIDNotification()
context.mergeChanges(fromContextDidSave: note) // heap buffer overflow!
try? self.storeHistoryToken(transaction.token)
}
}
}
I don't know what I could do to fix this code, since it seems pretty simple. It does happen when there's a lot of data being imported to Core Data through Batch Insertion Requests.
The output of the sanitizer into the debug terminal looks like this:
==59326==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6280004dbcff at pc 0x00010c71c8bd bp 0x7ff7b724b9d0 sp 0x7ff7b724b190
READ of size 15360 at 0x6280004dbcff thread T0
#0 0x10c71c8bc in wrap_memcpy+0x16c (libclang_rt.asan_iossim_dynamic.dylib:x86_64+0x1b8bc)
#1 0x7ff804aafd81 in fetchResultSetReallocCurrentRow+0x206 (CoreData:x86_64+0x1f2d81)
#2 0x7ff804a80dce in -[NSSQLiteConnection fetchResultSet:usingFetchPlan:]+0x317 (CoreData:x86_64+0x1c3dce)
#3 0x7ff804ab0ed1 in newFetchedRowsForFetchPlan_MT+0x471 (CoreData:x86_64+0x1f3ed1)
#4 0x7ff804bb0807 in _executeObjectFaultRequest+0x307 (CoreData:x86_64+0x2f3807)
#5 0x7ff804bb26e7 in _executeNewRowValuesForObjectFaultRequest+0xb6 (CoreData:x86_64+0x2f56e7)
#6 0x7ff804abe237 in -[NSSQLObjectFaultRequestContext executeRequestCore:]+0x14 (CoreData:x86_64+0x201237)
#7 0x7ff80491f370 in -[NSSQLStoreRequestContext executeRequestUsingConnection:]+0x1b0 (CoreData:x86_64+0x62370)
#8 0x7ff8049debfa in __52-[NSSQLDefaultConnectionManager handleStoreRequest:]_block_invoke+0x37 (CoreData:x86_64+0x121bfa)
#9 0x7ff804a729ca in __37-[NSSQLiteConnection performAndWait:]_block_invoke+0x1b (CoreData:x86_64+0x1b59ca)
#10 0x1123d2f5a in _dispatch_client_callout+0x7 (libdispatch.dylib:x86_64+0x4f5a)
#11 0x1123e4d71 in _dispatch_lane_barrier_sync_invoke_and_complete+0x83 (libdispatch.dylib:x86_64+0x16d71)
#12 0x7ff804a723d2 in -[NSSQLiteConnection performAndWait:]+0x8e (CoreData:x86_64+0x1b53d2)
#13 0x7ff8049deb13 in -[NSSQLDefaultConnectionManager handleStoreRequest:]+0xe3 (CoreData:x86_64+0x121b13)
#14 0x7ff804b68065 in -[NSSQLCoreDispatchManager routeStoreRequest:]+0x10e (CoreData:x86_64+0x2ab065)
#15 0x7ff804a32fd4 in -[NSSQLCore dispatchRequest:withRetries:]+0x84 (CoreData:x86_64+0x175fd4)
#16 0x7ff804a33ec3 in -[NSSQLCore newValuesForObjectWithID:withContext:error:]+0x137 (CoreData:x86_64+0x176ec3)
#17 0x7ff8049f8858 in __95-[NSPersistentStoreCoordinator(_NSInternalMethods) newValuesForObjectWithID:withContext:error:]_block_invoke+0x5b (CoreData:x86_64+0x13b858)
#18 0x7ff8049e64e6 in -[NSPersistentStoreCoordinator _routeLightweightBlock:toStore:]+0xeb (CoreData:x86_64+0x1294e6)
#19 0x7ff8049f85bd in -[NSPersistentStoreCoordinator(_NSInternalMethods) newValuesForObjectWithID:withContext:error:]+0x15c (CoreData:x86_64+0x13b5bd)
#20 0x7ff80496466f in _PFFaultHandlerLookupRow+0x204 (CoreData:x86_64+0xa766f)
#21 0x7ff804966c69 in _PF_FulfillDeferredFault+0xd6 (CoreData:x86_64+0xa9c69)
#22 0x7ff8049864a6 in _sharedIMPL_pvfk_core+0x8a (CoreData:x86_64+0xc94a6)
#23 0x7ff804976326 in _PF_Handler_Public_GetProperty+0xed (CoreData:x86_64+0xb9326)
#24 0x7ff800b9e985 in -[NSFunctionExpression expressionValueWithObject:context:]+0x6c2 (Foundation:x86_64+0x4a0985)
#25 0x7ff800b58fda in -[NSComparisonPredicate evaluateWithObject:substitutionVariables:]+0x146 (Foundation:x86_64+0x45afda)
#26 0x7ff800b5c0c0 in -[NSCompoundPredicateOperator evaluatePredicates:withObject:substitutionVariables:]+0x116 (Foundation:x86_64+0x45e0c0)
#27 0x7ff800b5ba14 in -[NSCompoundPredicate evaluateWithObject:substitutionVariables:]+0x109 (Foundation:x86_64+0x45da14)
#28 0x7ff804a18812 in -[NSFetchedResultsController _preprocessUpdatedObjects:insertsInfo:deletesInfo:updatesInfo:sectionsWithDeletes:newSectionNames:treatAsRefreshes:]+0x1f0 (CoreData:x86_64+0x15b812)
#29 0x7ff804a19c48 in __82-[NSFetchedResultsController(PrivateMethods) _core_managedObjectContextDidChange:]_block_invoke+0x7d7 (CoreData:x86_64+0x15cc48)
#30 0x7ff8049a0112 in developerSubmittedBlockToNSManagedObjectContextPerform+0x96 (CoreData:x86_64+0xe3112)
#31 0x7ff80499fffc in -[NSManagedObjectContext performBlockAndWait:]+0xc4 (CoreData:x86_64+0xe2ffc)
#32 0x7ff804a1945e in -[NSFetchedResultsController _core_managedObjectContextDidChange:]+0x6d (CoreData:x86_64+0x15c45e)
#33 0x7ff80035964a in __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__+0x88 (CoreFoundation:x86_64+0x5664a)
#34 0x7ff800359580 in ___CFXRegistrationPost_block_invoke+0x55 (CoreFoundation:x86_64+0x56580)
#35 0x7ff80AddressSanitizer report breakpoint hit. Use 'thread info -s' to get extended information about the report.
(lldb) thread info -s
thread #1: tid = 0x1361ef, 0x000000010c751210 libclang_rt.asan_iossim_dynamic.dylib`__asan::AsanDie(), queue = 'SQLQueue 0x613000128340 for MyAppModel.sqlite', stop reason = Heap buffer overflow
{
"access_size": 15360,
"access_type": 0,
"address": 108301900430591,
"description": "heap-buffer-overflow",
"instrumentation_class": "AddressSanitizer",
"pc": 4503750845,
"stop_type": "fatal_error"
}
Can anyone help me make sense of this?
Calloc is called from some signal handler in my production code. I know that it is not safe to call calloc from a signal handler.
I need to find the point where is this done in my production code, but backtrace is of no help. Please guide
print $_siginfo.si_signo
I know above tells the signal number of the current signal. But that does not help me to find the point.
Following is the call stack, the stacktrace just below calloc is of no use.
#26 0x00002aaaaaccfec9 in calloc () from
#27 0x00000000047c522f in Ox4074f61149c39b9f ()
#28 0x00000000047aa93a in Ox4074ff7b773cfdf6 ()
#29 0x000000000468c981 in Ox4074ea3d1f5df78f ()
#30 0x000000000468e667 in ?? ()
#31 0x000000000468f2dd in ?? ()
#32 0x0000000004686f2c in Ox4074ea18721a0b5d ()
#33 0x00000000047b3727 in Ox4075027a04de15e3 ()
#34 0x00000000047efdd4 in ?? ()
#35 <signal handler called>
#36 0x00002aaabc5d0332 in __lll_lock_wait () from /lib64/libpthread.so.0
#37 0x00002aaabc5cb5d8 in _L_lock_854 () from /lib64/libpthread.so.0
#38 0x00002aaabc5cb4a7 in pthread_mutex_lock () from
/lib64/libpthread.so.0
Please guide
The most likely explanation for the stack trace you observed: you are analyzing a core dump from production machine on a different machine, and using the wrong system libraries to do so.
See this answer on how to do this properly.
My application implements the NSApplicationDelegate method -application:openFile: in its app delegate. There seems to be a problem new to OS 10.8 where after receiving the apple event to open a file, five seconds pass before some kind of internal timeout is hit, a response is sent to the apple event, and the calling app (e.g., applescript editor) proceeds.
In detail, here's what I see. After application:openFile: returns YES, I can see that -[NSAppleEventManager suspendCurrentAppleEvent] is called:
#0 0x00007fff8f07b5ed in -[NSAppleEventManager suspendCurrentAppleEvent] ()
#1 0x00007fff8f90458e in -[NSApplication(NSAppleEventHandling) _handleAEOpenDocumentsForURLs:] ()
#2 0x00007fff8f903b55 in -[NSApplication(NSScripting) handleOpenScriptCommand:] ()
#3 0x00007fff90770cac in __invoking___ ()
#4 0x00007fff90770b47 in -[NSInvocation invoke] ()
#5 0x00007fff90770d19 in -[NSInvocation invokeWithTarget:] ()
#6 0x00007fff8f10121c in -[NSScriptCommand _sendToRemainingReceivers] ()
#7 0x00007fff8f1014c6 in -[NSScriptCommand executeCommand] ()
#8 0x00007fff8f11983a in -[NSScriptingAppleEventHandler handleCommandEvent:withReplyEvent:] ()
#9 0x00007fff8f00635b in -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] ()
#10 0x00007fff8f0061bd in _NSAppleEventManagerGenericHandler ()
#11 0x00007fff93844f68 in aeDispatchAppleEvent ()
#12 0x00007fff93844dc9 in dispatchEventAndSendReply ()
#13 0x00007fff93844c89 in aeProcessAppleEvent ()
#14 0x00007fff8c141e29 in AEProcessAppleEvent ()
#15 0x00007fff8f6b22a6 in _DPSNextEvent ()
#16 0x00007fff8f6b1862 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#17 0x00007fff8f6a8c03 in -[NSApplication run] ()
#18 0x00007fff8f64d656 in NSApplicationMain ()
#19 0x0000000100001d7c in main (argc=3, argv=0x7fff5fbff7c0) at main.m:23
After five seconds, -[NSAppleEventManager resumeWithSuspensionID:] is called:
#0 0x00007fff8f07b6be in -[NSAppleEventManager resumeWithSuspensionID:] ()
#1 0x00007fff8f90585f in __block_global_1 ()
#2 0x00007fff9073a4b4 in __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ ()
#3 0x00007fff90739fcd in __CFRunLoopDoTimer ()
#4 0x00007fff9071f7b9 in __CFRunLoopRun ()
#5 0x00007fff9071edd2 in CFRunLoopRunSpecific ()
#6 0x00007fff8c137774 in RunCurrentEventLoopInMode ()
#7 0x00007fff8c137512 in ReceiveNextEventCommon ()
#8 0x00007fff8c1373a3 in BlockUntilNextEventMatchingListInMode ()
#9 0x00007fff8f6b1fa3 in _DPSNextEvent ()
#10 0x00007fff8f6b1862 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#11 0x00007fff8f6a8c03 in -[NSApplication run] ()
#12 0x00007fff8f64d656 in NSApplicationMain ()
#13 0x0000000100001d7c in main (argc=3, argv=0x7fff5fbff7c0) at main.m:23
Evidently, some kind of timeout was reached since resumeWithSuspensionID: is called from a timer and it is always very close to 5 second between my application:openFile: method returning and resumeWithSuspensionID: being called.
After this, things proceed normally: whatever applescript invoked "open file ..." continues at the next statement. There don't appear to be any errors--it just times out waiting for something.
I turned on AEDebugSends and AEDebugReceives. Here's what I see up to the timeout:
Aug 23 21:26:05 mymachine.local iTerm[4819] <Info>: _aeMachPortCallback(), got a message, # 0x0x7fff5fbfe1f0 size=972 info=0 (_aeMachPortCallback()/AEMach.cp #643) main
Aug 23 21:26:05 mymachine.local iTerm[4819] <Notice>: AE2000 (4819: Received an event:
------oo start of event oo------
aevt('aevt'\'odoc' transactionID=0 returnID=3711 sourcePSN=[0x0,6f06f "AppleScript Editor"] eventSource=3 sourceUID=501 sourceGID=20 sourceEUID=501 sourceEGID=20 sourcePID=3551 auditToken=[501,501,20,501,20,3551,100005,3551]{ '----':obj ('obj '{ 'form':enum('name'), 'want':type('file'), 'seld':utxt('utxt'(TEXT("Macintosh HD:Users:georgen:Library:Application Support:iTerm:version.txt"))), 'from':null() }), &'csig':magn(65536) })
------oo end of event oo------
(aeProcessIncomingEvent()/AEMach.cp #1445) main
2012-08-23 21:26:05.451 iTerm[4819:707] Command: NSCoreSuite.Open
Direct Parameter: /Users/georgen/Library/Application Support/iTerm/version.txt
Receivers: (null)
Arguments: {
"" = "/Users/georgen/Library/Application Support/iTerm/version.txt";
}
Then, after the timeout I see this (which is logged because I also set NSScriptingDebugLogLevel to 1):
2012-08-23 21:26:36.347 iTerm[4819:707] Result: <null>
My openFile: method doesn't do anything besides return YES. Is there something else I need to do to make the framework happy?
Why are none of the addresses from the gdb backtrace output present in the otx (otool) assembly output of this program? I was under the impression that they corresponded to the offsets in my otool asm output... I apologize if this is a silly question.
(gdb) bt
#0 0x9a4e1aa2 in __semwait_signal ()
#1 0x9a4e175e in _pthread_cond_wait ()
#2 0x9a4e12b1 in pthread_cond_timedwait$UNIX2003 ()
#3 0x0d8f3f87 in unregister_ShockwaveFlash ()
#4 0x0d45cf19 in dyld_stub_write$UNIX2003 ()
#5 0x0d6f9d7e in dyld_stub_write$UNIX2003 ()
#6 0x0d72db5a in dyld_stub_write$UNIX2003 ()
#7 0x0d72e24c in dyld_stub_write$UNIX2003 ()
#8 0x0d8eb5a2 in unregister_ShockwaveFlash ()
#9 0x0d95b9c9 in unregister_ShockwaveFlash ()
#10 0x9277df60 in CAOpenGLLayerDraw ()
#11 0x9277e897 in -[CAOpenGLLayer _display] ()
#12 0x92503ef1 in CALayerDisplayIfNeeded ()
#13 0x925032bc in CA::Context::commit_transaction ()
#14 0x92502f04 in CA::Transaction::commit ()
#15 0x958a1dd2 in __CFRunLoopDoObservers ()
#16 0x9585d3ea in CFRunLoopRunSpecific ()
#17 0x9585d1f1 in CFRunLoopRunInMode ()
#18 0x95530e04 in RunCurrentEventLoopInMode ()
#19 0x95530bb9 in ReceiveNextEventCommon ()
#20 0x956b9084 in _AcquireNextEvent ()
#21 0x956aed40 in RunApplicationEventLoop ()
#22 0x100083e2 in fkDecode ()
#23 0x10026167 in fkCall ()
#24 0x100081f6 in fkDecode ()
#25 0x10026167 in fkCall ()
#26 0x100081f6 in fkDecode ()
#27 0x10003be5 in fkRunMain ()
#28 0x00001e84 in main ()</code>
The followings may help you :
Variables defined in Global and Local Scope may have different address due to Address Space Layout Randomization (ASLR)
Heap allocated variables may have different address due to ASLR, if your program is multi-threaded
On Linux GDB is by default disables ASLR. Try it on Mac disabling ASLR.
I have a release version server process running under linux 64-bit systems. It got crashed and left a coredump file. I use gdb to debug it like this:
gdb svr coredump file
And got the following backtraces:
(gdb) where
#0 0x0000000000432691 in ?? ()
#1 0x00002b07655a50cc in ?? ()
#2 0x00002b07655a50c4 in ?? ()
#3 0x00007fff9fade920 in ?? ()
#4 0x0000000000439db3 in ?? ()
#5 0x00007fff9fade910 in ?? ()
#6 0x00007fff9fade910 in ?? ()
#7 0x00007fff9fade8e0 in ?? ()
#8 0x00000000004663e2 in ?? ()
#9 0x00007fff9fade910 in ?? ()
#10 0x00007fff9fade910 in ?? ()
#11 0x00007fff9fade930 in ?? ()
#12 0x0000000000605108 in ?? ()
#13 0x00002b07655a274c in ?? ()
#14 0x0000000000ebc678 in ?? ()
#15 0x169f49f100000001 in ?? ()
#16 0x00000000021dc6c0 in ?? ()
#17 0x00002b07655a284c in ?? ()
#18 0x00002b07655a27dc in ?? ()
#19 0x00007fff9fade960 in ?? ()
#20 0x000000000043a03b in ?? ()
#21 0x00007fff9fade960 in ?? ()
#22 0x0000000000994d02 in ?? ()
#23 0x00000000000ecd57 in ?? ()
#24 0x00002b07655a274c in ?? ()
#25 0x00002b07655a274c in ?? ()
#26 0x00002b07655a27dc in ?? ()
#27 0x00007fff9fade980 in ?? ()
#28 0x000000000060a5eb in ?? ()
#29 0x000000009fadeb50 in ?? ()
#30 0x00002b07655a274c in ?? ()
#31 0x00007fff9fade9d0 in ?? ()
#32 0x000000000060a8f0 in ?? ()
#33 0x00007fff9fadeb50 in ?? ()
#34 0x00007fff9fadea10 in ?? ()
#35 0x00002b07655a274c in ?? ()
#36 0x00007fff9fadea10 in ?? ()
#37 0x000000009fade9d0 in ?? ()
#38 0x00007fff9fadeb58 in ?? ()
#39 0x0000000000000000 in ?? ()
The address info can not be analyzed by addr2line, what is the problem and how can I find what is the root cause of coredump?
Thanks!
Are you running GDB on the machine on which the core dump was produced?
For GDB to correctly reconstruct the crash stack trace, it must have access to exact binaries which were used at the time of the crash (or you get garbage).
The easiest way to achieve this is to analyze the core on the machine on which it was produced. If that's not feasible, you must copy all shared libraries to e.g. /tmp/solib-on-server/ (preserve the path, so e.g. /lib/libc.so.6 goes into /tmp/solib-on-server/lib/libc.so.6), then use GDB set solib-absolute-prefix /tmp/solib-on-server command before loading the core. E.g.
gdb -ex 'set solib-absolute-prefix /tmp/solib-on-server' \
-ex 'core /path/to/core' /path/to/executable
Its very hard to debug programs without debug symbols. As you are using release version of the application the core dump will not contain any debug information.
I am not sure, but if you can correlate the stack trace with the ".map" file of your application you could be getting somewhere. If I was in your position I would have recompiled the app with debug symbols(-g compiler flag) and then proceed on debugging.
You can use below commands in gdb to set shared library path.
set solib-search-path [Directories]
[Directories] : paths separated by colon(Ex /usr/lib:/usr/lib64)
show solib-search-path
These two commands helped me to get some info.