pdf of this jumpstart:
notes-mdb-jumpstart.pdf
invoke:
mdb rps core
mdb rps -l
UMEM_DEBUG=default UMEM_LOGGING=transaction LD_PRELOAD=libumem.so.1 ./rps
side-note try: `pstack core`
attach to process:
mdb -p
help/info:
::help
::dcmds
::formats
::walkers
some basic control:
::quit / $q - quit
::stack / $c / $C - print current stack trace
::walk thread |::findstack
examine memory:
/ - display memory in format
e.g.
1025c5a50/10X - show 10 32bit hex integers:
1025c5a50/s - show as string
1025c5a50::print [type] - print as type (if type can be determined (needs debug info in binary))
0x70a9ae00,5/KKn
::dis - disassembly
libumem use:
sun article on using libumem
::findleaks
::umem_status
::umem_verify
::umem_log
::umem_log!wc -l
::walk umem_log |::bufctl_audit
::walk umem_log |::bufctl_audit ! cat > file
::walk umem_alloc_12288 |::bufctl_audit
0x70ae3200$umausers-3.log
::walk umem_alloc_96 | ::map '*(.+60+8)' | ::bufctl_audit
::umem_cache ! grep umem_alloc_96
10020c028::walk bufctl | ::bufctl_audit
113429240::whatis
control running process:
# no symbols available until process loaded so set breakpoint in main
::bp main
::run -c -l
::nm !grep UseridCache|grep Add
::bp _ZN11UseridCache3AddEjP3RAIP4IMSI
::events
brief gdb to mdb list:
a gdb to mdb list
run control
:A / [pid]::attach [core|pid]
:R / ::release -a
:r / ::run args
:c / ::cont [sig]
:e / ::next [sig]
:s :u / ::step [over|out] [sig]
$c / [addr]::stack [count]
watch/break-points (events)
:b / [addr]::bp .. stuff .. [cmd] [-n count] sym
:a :p :w / addr[,len]::wp
:t / [signal]::sigbp
:d / [addr]::delete [id|all]
$b / ::events [-av]
e.g.
# if umem causes a core then the first thing to run is ::umem_status
# finding leaks (attach to process using mdb -p
> ::findleaks
CACHE LEAKED BUFCTL CALLER
00000001025e2b28 2846 00000001026627f0 _Z18DecodeIupsUserplIpPP6CNlPdu+0xac
00000001025e2b28 1 0000000110f2fce0 _Z18DecodeIupsUserplIpPP6CNlPdu+0xac
00000001025e2b28 2846 00000001026628d0 _Z19DecodeIupsUserplUdpPP16CIupsUserplIpPdu+0xc8
00000001025e2b28 1 0000000110f2fdc0 _Z19DecodeIupsUserplUdpPP16CIupsUserplIpPdu+0xc8
----------------------------------------------------------------------
Total 5694 buffers, 273312 bytes
> 00000001026627f0::bufctl_audit
ADDR BUFADDR TIMESTAMP THR LASTLOG CONTENTS
CACHE SLAB NEXT DEPTH
00000001026627f0 0000000102660ba0 64b785c211775a 1 00000001025d98c0 0000000000000000
00000001025e2b28 00000001025fbc30 000000010fe62720 7
libumem.so.1`umem_cache_alloc+0x228
libumem.so.1`umem_alloc+0x3c
libumem.so.1`malloc+0x28
_Z18DecodeIupsUserplIpPP6CNlPdu+0xac
_Z13process_blockv+0x6ac
main+0x13d4
_start+0x7c
# examining libumem buffer redzones and debug info
::walk umem_alloc_4096
1025b6240+0x1000/20X
10258cab0::bufctl_audit
1025b6240+0x1008/1J | ::bufctl_audit
* <----------- mi.malloc_size --->
* <----------- cp.cache_bufsize ------------------>
* <----------- cp.cache_chunksize -------------------------------->
* +-------+-----------------------+---------------+---------------+
* |/tag///| mallocsz |/round-off/////|/debug info////|
* +-------+---------------------------------------+---------------+
# using ::umem_verify
::umem_verify
>> umem_alloc_16 3d6c8 1 corrupt buffer
3d6c8::umem_verify
>> Summary for cache 'umem_alloc_16'
>> buffer 49fe0 (free) seems corrupted, at 0
49fe0/10X
>> 0x49fe0: deadbeef deadbeef 30313233 3400beef
>> feedface feedface 54780 f4ebb36e
>> deadbeef deadbeef
> ::umalog |::grep 49fe0