Peeter Joot's (OLD) Blog.

Math, physics, perl, and programming obscurity.

AIX dbx has gotten clever with lwarx/stwcx. handling somewhere along the way.

Posted by peeterjoot on July 15, 2010

Eons ago when I rewrote the internals of the DB2 reader-writer mutex (what we call a shared latch in development) I’d observed that instruction stepping through the atomic operation would cause the atomic operation to retry. This is visible on PowerPC since compare and swap and any other “atomic” operation are synthetic, built on the load-locked, store-conditional paradigm. Here’s an example of the instructions one gets for a fetch-and-or sequence from a call to the builtins.h routine:

[10] stopped in main at line 187 in file "ossPowerPcVACPP.h" ($t1)
  187      return __fetch_and_or( (volatile unsigned int *)pAtomic, value ) ;
(dbx) listi
0x100000948 (main+0x108) 7ca01828       lwarx   r5,r0,r3
0x10000094c (main+0x10c) 7ca40378          or   r4,r5,r0
0x100000950 (main+0x110) 7c80192d      stwcx.   r4,r0,r3
0x100000954 (main+0x114) 40a2fff4         bne   0x100000948 (main+0x108)
0x100000958 (main+0x118) 74a01000      andis.   r0,r5,0x1000
0x10000095c (main+0x11c) 40820034         bne   0x100000990 (main+0x150)
0x100000960 (main+0x120) 4c00012c       isync

We have a load (lwarx), change (or), and store (stwcx.) sequence that implements the atomic operation. The idea is that if the memory is changed by some other mechanism (ie: another thread or process writing to the mutex memory or a near enough adjacent location), a memory “reservation” is lost, and the sequence has to be restarted at the load. This example is for a mutex acquision, which you can tell by the isync memory barrier following the successful mutex lock operation.

Today, it was pointed out to me that instruction stepping through this code didn’t cause the atomic operation to retry internally. Sure enough, I see the same thing:

[10] tstopi at 0x100000948 (main+0x108)  for $t1
(dbx) c
[10] stopped in main at line 187 in file "ossPowerPcVACPP.h" ($t1)
  187      return __fetch_and_or( (volatile unsigned int *)pAtomic, value ) ;
(dbx) listi
0x100000948 (main+0x108) 7ca01828       lwarx   r5,r0,r3
0x10000094c (main+0x10c) 7ca40378          or   r4,r5,r0
0x100000950 (main+0x110) 7c80192d      stwcx.   r4,r0,r3
0x100000954 (main+0x114) 40a2fff4         bne   0x100000948 (main+0x108)
0x100000958 (main+0x118) 74a01000      andis.   r0,r5,0x1000
0x10000095c (main+0x11c) 40820034         bne   0x100000990 (main+0x150)
0x100000960 (main+0x120) 4c00012c       isync
0x100000964 (main+0x124) e88100b2         lwa   r4,0xb0(r1)
0x100000968 (main+0x128) 387e0250        addi   r3,0x250(r30)
0x10000096c (main+0x12c) 48000175          bl   0x100000ae0 (printf)
(dbx) p $r3
0x0fffffffffffea40
(dbx) p $r5
0x09001000accca260
(dbx) p $r0
0x0000000010000000
(dbx) stepi
stopped in main at 0x100000954 ($t1)
0x100000954 (main+0x114) 40a2fff4         bne   0x100000948 (main+0x108)
(dbx) p $r3
0x0fffffffffffea40
(dbx) p $r4
0x0000000010000000
(dbx) 0x0fffffffffffea40/10
0x0fffffffffffea40:  10000000 00000000 00000000 e0ddf00d
0x0fffffffffffea50:  0fffffff ffffea40 0fffffff ffffea48
0x0fffffffffffea60:  0fffffff ffffea40

BUT. Look carefully. I’d set the breakpoint at the lwarx operation, and stepi popped me right past the stwcx. to the branch conditional following the atomic store. Somewhere since I first personally observed the quantum effect of the debugger with this sort of debugging attempt (ie: looking at it changes it), the debugger guys got really clever in their handling of this. Did some customer actually complain about this, and they actually went to the effort of modifying the debugger to disassemble the code following the lwarx and only allow instruction stepping to a “safe” location?

Funky and amusing!

Advertisements

One Response to “AIX dbx has gotten clever with lwarx/stwcx. handling somewhere along the way.”

  1. […] by peeterjoot on July 23, 2010 In a previous post on an AIX dbx quirk with the stepi command, the following mutex acquision fragment was used as an […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: