Peeter Joot's (OLD) Blog.

Math, physics, perl, and programming obscurity.

stack layout on amd64

Posted by peeterjoot on November 19, 2010

I’ve got a raw stackdump to deal with in a stack corruption issue, but didn’t know the stack layout for the linux amd64 ABI. On AIX, we find nice pairs of stackframe-address/instruction-addresses, and kind of expected to see that on linux too, but didn’t. To see how this works, I compiled a simple program, and walked through a call in the debugger.

Prior to a call instruction, I’ve got the following in my stack:

(gdb) x/5gx $rsp
0x7fffffffcac8: 0x0000000000400845      0x00002aaaaabc7000
0x7fffffffcad8: 0x0000000000400834      0x00007fff00000007
0x7fffffffcae8: 0x00007fff00000008

And the program is sitting at the following location:

(gdb) p /x $rip
$3 = 0x4006fc
(gdb) disassemble
Dump of assembler code for function _Z4bar1v:
0x00000000004006fc :        sub    $0x8,%rsp
0x0000000000400700 :        callq  0x40070c 
0x0000000000400705 :        add    $0x8,%rsp
0x0000000000400709 :       retq
End of assembler dump.

One more instruction step gets me to the call point:

(gdb) stepi
0x0000000000400700 in bar1() ()

And my stack contents now contain:

(gdb) p $rsp
$4 = (void *) 0x7fffffffcac0
(gdb) x/5gx $rsp
0x7fffffffcac0: 0x00002aaa00000002      0x0000000000400845
0x7fffffffcad0: 0x00002aaaaabc7000      0x0000000000400834
0x7fffffffcae0: 0x00007fff00000007

The stack pointer had been decremented. Is this a stack frame allocated for the bar1() function itself, or a decrement in preparation for a save of the return address value?

One more instruction step gets me into the new function. This appears to automatically decrement my stack pointer $rsp 8 more bytes, and I now have the return address (0x400705) in the stack in the first 64-bits:

(gdb) stepi
0x000000000040070c in bar2() ()
(gdb) p /x $rip
$5 = 0x40070c

(gdb) x/5gx $rsp
0x7fffffffcab8: 0x0000000000400705      0x00002aaa00000002
0x7fffffffcac8: 0x0000000000400845      0x00002aaaaabc7000
0x7fffffffcad8: 0x0000000000400834

Does the linux (amd64) ABI require every function to allocate a stack frame, even if they don’t use it? I see this in bar1()’s code despite compiling with -O. It appears that the ret instruction also pops from the stack, incrementing $rsp as it branches to that address. It also looks like what we require to find out return address is to look at the pre-decrement value of $rsp (or calculate that), so to navigate the stack manually in a corrupted stack dump, we’ve got to disassemble to know where to look for the next return address. That’s much messier than on AIX where we can look for the longest chain of stackframe/saved-link-addresses to attempt to find a pre-corruption point in the code to try to pinpoint where things went wrong.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: