A gdb tip for threaded debugging. scheduler locking.

Threaded debugging can behave oddly on a number of platforms. In particular the next command can behave oddly. Here’s an example:

(gdb) n
29375         m_pConnEntry->sqleCaCeGetCachedRegisteredMemory( &m_mem, &m_pRegistration ) ;
(gdb) n
[New Thread 47757037398336 (LWP 5917)]

Breakpoint 2, SAL_SA_HANDLE::SAL_ReadSAList (this=0x2b6f54b75420, pSaPageName=0x2b6f4ebfcfb0, ecfFunc=423100625,
    ecfProbe=4262, pBuffer=0x2b6f4ebece70 "01", pEntryCount=0x2b6f4ebfcfa4, bReadAllSAs=true,
    pSALReadSAListIterator=0x2b6f4ebfce70) at sqle_ca_cmd.C:29629
29629      pdTraceExit( SQLT_SAL_SA_HANDLE__SAL_ReadSAList, rc ) ;
(gdb) q

I was in the function SAL_ReadSAList, single stepping line-by-line (I happened to also have a breakpoint set at the end of the function, but didn’t expect to hit it) … then BAM, a new thread got started up and my next command somehow gets interpreted by the debugger as “next, but maybe something else random for another thread too”. Pain in the butt, and now I’ve lost the context I wanted to debug, and have to start over.

On platforms that support it, the gdb scheduler locking feature works nicely. Here’s an example:

(gdb) b 4233
Breakpoint 1 at 0x2b3bbe7285eb: file testca.C, line 4233.
(gdb) signal SIGCONT
Continuing with signal SIGCONT.

Breakpoint 1, SA_TESTS (n1=1, flags=1, init=18446744073709551615, next=18446744073709551615) at testca.C:4239
4239                                                                           &i ) ;
(gdb) set scheduler-locking on
(gdb) n
4241          if ( rc )

Get the debugger to where I wanted (which in this case included raising a SIGCONT to get out of a pause() call where I’d forced things to stop). Once I’m there, turn on the scheduler lock so only my thread runs, then can step through my code without worrying about other threads interfering.

Note that this can also be a good way to lock up a threaded program. If your thread blocks on something like a pthread_mutex, and you haven’t let other threads run, you’ll get it stuck there without intervention (i.e. set scheduler-locking off) when you are done.

gdb signal SIGCONT and pause()

Anybody used to debugging on AIX is probably familiar with the convienence of the handy pause() function. Put a call to this in your program where you want to get the debugger attached, and let it run. Once you attach dbx you can ‘next’ past the pause() function to the code you want to debug.

This doesn’t work on Linux, and I found out the magic trick today. Attach gdb, find your thread, and set a breakpoint at a good spot past the pause() call. Then, do:

(gdb) signal SIGCONT

Voila. You’ll then hit your breakpoint, and can find the bug or walk through the code and learn it’s behaviour, …

Note that unlike the kill command, gdb doesn’t take cont or sigcont as allowable variations of SIGCONT.

EDIT: I can no longer get this to work with gdb 7.0. If anybody knows why I would be interested.