A gdb tip for threaded debugging. scheduler locking.
Posted by peeterjoot on August 25, 2009
Threaded debugging can behave oddly on a number of platforms. In particular the next command can behave oddly. Here’s an example:
(gdb) n 29375 m_pConnEntry->sqleCaCeGetCachedRegisteredMemory( &m_mem, &m_pRegistration ) ; (gdb) n [New Thread 47757037398336 (LWP 5917)] Breakpoint 2, SAL_SA_HANDLE::SAL_ReadSAList (this=0x2b6f54b75420, pSaPageName=0x2b6f4ebfcfb0, ecfFunc=423100625, ecfProbe=4262, pBuffer=0x2b6f4ebece70 "01", pEntryCount=0x2b6f4ebfcfa4, bReadAllSAs=true, pSALReadSAListIterator=0x2b6f4ebfce70) at sqle_ca_cmd.C:29629 29629 pdTraceExit( SQLT_SAL_SA_HANDLE__SAL_ReadSAList, rc ) ; (gdb) q
I was in the function SAL_ReadSAList, single stepping line-by-line (I happened to also have a breakpoint set at the end of the function, but didn’t expect to hit it) … then BAM, a new thread got started up and my next command somehow gets interpreted by the debugger as “next, but maybe something else random for another thread too”. Pain in the butt, and now I’ve lost the context I wanted to debug, and have to start over.
On platforms that support it, the gdb scheduler locking feature works nicely. Here’s an example:
(gdb) b 4233 Breakpoint 1 at 0x2b3bbe7285eb: file testca.C, line 4233. (gdb) signal SIGCONT Continuing with signal SIGCONT. Breakpoint 1, SA_TESTS (n1=1, flags=1, init=18446744073709551615, next=18446744073709551615) at testca.C:4239 4239 &i ) ; (gdb) set scheduler-locking on (gdb) n 4241 if ( rc )
Get the debugger to where I wanted (which in this case included raising a SIGCONT to get out of a pause() call where I’d forced things to stop). Once I’m there, turn on the scheduler lock so only my thread runs, then can step through my code without worrying about other threads interfering.
Note that this can also be a good way to lock up a threaded program. If your thread blocks on something like a pthread_mutex, and you haven’t let other threads run, you’ll get it stuck there without intervention (i.e. set scheduler-locking off) when you are done.