I asked this question:Running `sudo /sbin/service mysqld start` causes system to crashIn the comments of one of the answers, a very helpful individual is asking for more information regarding how much memory processes were using when oom-killer is invoked.The output of dmesg is ~9000 lines, and hundreds of thousands of characters, though.An example of the output that I see multiple times throughout the output is this:Out of memory: Killed process 21000, UID 48, (httpd).mysqld invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0Call Trace: [<ffffffff802c1b64>] out_of_memory+0x8b/0x203 [<ffffffff8020fa5d>] __alloc_pages+0x27f/0x308 [<ffffffff802139dd>] __do_page_cache_readahead+0xc8/0x1af [<ffffffff8021424e>] filemap_nopage+0x14c/0x360 [<ffffffff80208e9d>] __handle_mm_fault+0x444/0x144f [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff80266d94>] do_page_fault+0xf72/0x131b [<ffffffff802456a8>] sys_rt_sigreturn+0x327/0x35a [<ffffffff8026393d>] _spin_lock_irq+0x9/0x14 [<ffffffff802296ed>] do_sigaction+0x18c/0x1a1 [<ffffffff8025f82b>] error_exit+0x0/0x6eMem-info:DMA per-cpu:cpu 0 hot: high 0, batch 1 used:0cpu 0 cold: high 0, batch 1 used:0DMA32 per-cpu:cpu 0 hot: high 186, batch 31 used:30cpu 0 cold: high 62, batch 15 used:51Normal per-cpu: emptyHighMem per-cpu: emptyFree pages:        4748kB (0kB HighMem)Active:114975 inactive:0 dirty:0 writeback:0 unstable:0 free:1187 slab:4839 mapped-file:541 mapped-anon:114505 pagetables:1332DMA free:2004kB min:48kB low:60kB high:72kB active:540kB inactive:0kB present:9076kB pages_scanned:5699645 all_unreclaimable? yeslowmem_reserve[]: 0 489 489 489DMA32 free:2744kB min:2804kB low:3504kB high:4204kB active:459360kB inactive:0kB present:500960kB pages_scanned:11998796 all_unreclaimable? yeslowmem_reserve[]: 0 0 0 0Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? nolowmem_reserve[]: 0 0 0 0HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? nolowmem_reserve[]: 0 0 0 0DMA: 1*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kBDMA32: 12*4kB 3*8kB 1*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2744kBNormal: emptyHighMem: empty586 pagecache pagesSwap cache: add 0, delete 0, find 0/0, race 0+0Free swap  = 0kBTotal swap = 0kBFree swap:            0kB131072 pages of RAM5993 reserved pages5581 pages shared0 pages swap cachedklogd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0Call Trace: [<ffffffff802c1b64>] out_of_memory+0x8b/0x203 [<ffffffff8020fa5d>] __alloc_pages+0x27f/0x308 [<ffffffff802139dd>] __do_page_cache_readahead+0xc8/0x1af [<ffffffff8021424e>] filemap_nopage+0x14c/0x360 [<ffffffff80208e9d>] __handle_mm_fault+0x444/0x144f [<ffffffff80263929>] _spin_lock_irqsave+0x9/0x14 [<ffffffff80263929>] _spin_lock_irqsave+0x9/0x14 [<ffffffff80251144>] finish_wait+0x32/0x5d [<ffffffff80266d94>] do_page_fault+0xf72/0x131b [<ffffffff802ff918>] kmsg_read+0x3a/0x44 [<ffffffff8025f82b>] error_exit+0x0/0x6eMem-info:DMA per-cpu:cpu 0 hot: high 0, batch 1 used:0cpu 0 cold: high 0, batch 1 used:0DMA32 per-cpu:cpu 0 hot: high 186, batch 31 used:45cpu 0 cold: high 62, batch 15 used:51Normal per-cpu: emptyHighMem per-cpu: emptyFree pages:        4748kB (0kB HighMem)Active:114975 inactive:0 dirty:0 writeback:0 unstable:0 free:1187 slab:4824 mapped-file:541 mapped-anon:114505 pagetables:1332DMA free:2004kB min:48kB low:60kB high:72kB active:540kB inactive:0kB present:9076kB pages_scanned:6531037 all_unreclaimable? yeslowmem_reserve[]: 0 489 489 489DMA32 free:2744kB min:2804kB low:3504kB high:4204kB active:459360kB inactive:0kB present:500960kB pages_scanned:36231106 all_unreclaimable? yeslowmem_reserve[]: 0 0 0 0Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? nolowmem_reserve[]: 0 0 0 0HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? nolowmem_reserve[]: 0 0 0 0DMA: 1*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kBDMA32: 12*4kB 3*8kB 1*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2744kBNormal: emptyHighMem: empty586 pagecache pagesSwap cache: add 0, delete 0, find 0/0, race 0+0Free swap  = 0kBTotal swap = 0kBFree swap:            0kB131072 pages of RAM5993 reserved pages5581 pages shared0 pages swap cachedOut of memory: Killed process 21001, UID 48, (httpd).sudo invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0Call Trace: [<ffffffff802c1b64>] out_of_memory+0x8b/0x203 [<ffffffff8020fa5d>] __alloc_pages+0x27f/0x308 [<ffffffff802139dd>] __do_page_cache_readahead+0xc8/0x1af [<ffffffff8021424e>] filemap_nopage+0x14c/0x360 [<ffffffff80208e9d>] __handle_mm_fault+0x444/0x144f [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff8020622a>] hypercall_page+0x22a/0x1000 [<ffffffff80266d94>] do_page_fault+0xf72/0x131b [<ffffffff8024901b>] skb_dequeue+0x48/0x50 [<ffffffff80254146>] unix_release_sock+0x19e/0x1fa [<ffffffff80261df5>] thread_return+0x6c/0x113 [<ffffffff80207116>] kmem_cache_free+0x84/0xd7 [<ffffffff80207116>] kmem_cache_free+0x84/0xd7 [<ffffffff8025f82b>] error_exit+0x0/0x6eMem-info:DMA per-cpu:cpu 0 hot: high 0, batch 1 used:0cpu 0 cold: high 0, batch 1 used:0DMA32 per-cpu:cpu 0 hot: high 186, batch 31 used:165cpu 0 cold: high 62, batch 15 used:48Normal per-cpu: emptyHighMem per-cpu: emptyFree pages:        4760kB (0kB HighMem)Active:114996 inactive:0 dirty:0 writeback:0 unstable:0 free:1190 slab:4821 mapped-file:541 mapped-anon:114468 pagetables:1225DMA free:2004kB min:48kB low:60kB high:72kB active:540kB inactive:0kB present:9076kB pages_scanned:7319613 all_unreclaimable? yeslowmem_reserve[]: 0 489 489 489DMA32 free:2756kB min:2804kB low:3504kB high:4204kB active:459444kB inactive:0kB present:500960kB pages_scanned:13331345 all_unreclaimable? yeslowmem_reserve[]: 0 0 0 0Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? nolowmem_reserve[]: 0 0 0 0HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? nolowmem_reserve[]: 0 0 0 0DMA: 1*4kB 0*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kBDMA32: 5*4kB 8*8kB 1*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 2756kBNormal: emptyHighMem: empty586 pagecache pagesSwap cache: add 0, delete 0, find 0/0, race 0+0Free swap  = 0kBTotal swap = 0kBFree swap:            0kB131072 pages of RAM5993 reserved pages4629 pages sharedHow do I read this in a way that can help me perform this task:In the process list that gets dumped into dmesg at the time of the OOM  condition, one of the columns should include how much memory each process is using.  ...put that output in your question...
How do I read the output of `dmesg` to determine how much memory a process is using when oom-killer is invoked?
centos;dmesg;out of memory