Page MenuHomePhabricator
Files F630145

kern.log
Public
Actions

Authored By
joshuaspence
Jul 17 2015, 4:25 AM
Size
4 MB
Referenced Files
None
Subscribers
None

kern.log
View Options

This file is larger than 256 KB, so syntax highlighting was skipped.
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720648] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720652] php cpuset=/ mems_allowed=0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720655] Pid: 9173, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720656] Call Trace:
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720663] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720665] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720668] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720671] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720674] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720677] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720679] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720682] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720685] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720688] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720691] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720693] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720696] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720698] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720702] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720704] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720706] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720709] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720712] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720714] Mem-Info:
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720715] Node 0 DMA per-cpu:
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720717] CPU 0: hi: 0, btch: 1 usd: 0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720718] CPU 1: hi: 0, btch: 1 usd: 0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720719] Node 0 DMA32 per-cpu:
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720721] CPU 0: hi: 186, btch: 31 usd: 30
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720722] CPU 1: hi: 186, btch: 31 usd: 53
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720725] active_anon:932617 inactive_anon:1393 isolated_anon:0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720726] active_file:0 inactive_file:0 isolated_file:0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720726] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720727] free:4029 slab_reclaimable:2703 slab_unreclaimable:2697
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720728] mapped:1339 shmem:1405 pagetables:3164 bounce:0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720729] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720736] lowmem_reserve[]: 0 3772 3772 3772
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720739] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3730468kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5356kB shmem:5620kB slab_reclaimable:10812kB slab_unreclaimable:10788kB kernel_stack:784kB pagetables:12656kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:116 all_unreclaimable? yes
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720745] lowmem_reserve[]: 0 0 0 0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720748] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720754] Node 0 DMA32: 1433*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7780kB
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720761] 1434 total pagecache pages
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720762] 0 pages in swap cache
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720763] Swap cache stats: add 0, delete 0, find 0/0
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720764] Free swap = 0kB
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.720765] Total swap = 0kB
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726142] 985072 pages RAM
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726144] 24998 pages reserved
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726145] 3792 pages shared
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726147] 953274 pages non-shared
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726148] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726157] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726161] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726164] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726167] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726170] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726174] [ 664] 0 664 1815 123 1 0 0 dhclient3
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726177] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726180] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726184] [ 790] 101 790 63477 545 0 0 0 rsyslogd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726187] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726190] [ 831] 0 831 4226 41 0 0 0 atd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726194] [ 975] 0 975 6276 80 0 0 0 master
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726197] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726200] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726203] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726206] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726209] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726212] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726216] [15657] 0 15657 4777 60 0 0 0 cron
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726219] [ 654] 1012 654 70287 5737 1 0 0 php
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726222] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726225] [ 658] 1012 658 73718 10819 0 0 0 php
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726228] [ 678] 1012 678 70152 5559 0 0 0 php
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726231] [13468] 0 13468 89584 2524 0 0 0 php5-fpm
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726234] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726237] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726240] [ 9173] 1012 9173 956614 893396 0 0 0 php
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726243] [25666] 1012 25666 68618 5633 0 0 0 php
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726247] [15959] 108 15959 6792 69 0 0 0 pickup
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726250] [20529] 108 20529 6795 70 0 0 0 trivial-rewrite
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726253] [20530] 108 20530 11107 159 0 0 0 smtp
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726256] [20626] 108 20626 6818 74 0 0 0 cleanup
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726259] [20630] 108 20630 11107 157 0 0 0 smtp
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726262] [20634] 108 20634 11107 157 0 0 0 smtp
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726265] [20638] 108 20638 11107 142 1 0 0 smtp
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726268] [20642] 108 20642 11107 158 1 0 0 smtp
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726271] [20648] 108 20648 6792 69 0 0 0 scache
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726274] Out of memory: Kill process 9173 (php) score 932 or sacrifice child
Jul 12 07:29:31 ip-10-61-174-46 kernel: [1141066.726286] Killed process 9173 (php) total-vm:3826456kB, anon-rss:3573576kB, file-rss:8kB
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004159] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004164] php cpuset=/ mems_allowed=0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004167] Pid: 7531, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004169] Call Trace:
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004177] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004180] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004183] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004187] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004193] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004196] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004201] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004204] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004208] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004211] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004215] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004218] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004222] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004225] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004228] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004232] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004235] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004238] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004242] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004244] Mem-Info:
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004245] Node 0 DMA per-cpu:
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004248] CPU 0: hi: 0, btch: 1 usd: 0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004250] CPU 1: hi: 0, btch: 1 usd: 0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004251] Node 0 DMA32 per-cpu:
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004253] CPU 0: hi: 186, btch: 31 usd: 0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004255] CPU 1: hi: 186, btch: 31 usd: 0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004258] active_anon:933071 inactive_anon:1393 isolated_anon:0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004259] active_file:0 inactive_file:0 isolated_file:0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004260] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004261] free:4048 slab_reclaimable:2519 slab_unreclaimable:2627
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004262] mapped:1367 shmem:1405 pagetables:3058 bounce:0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004264] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004273] lowmem_reserve[]: 0 3772 3772 3772
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004277] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732284kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5620kB slab_reclaimable:10076kB slab_unreclaimable:10508kB kernel_stack:712kB pagetables:12232kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004285] lowmem_reserve[]: 0 0 0 0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004288] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004297] Node 0 DMA32: 886*4kB 183*8kB 38*16kB 9*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004307] 1467 total pagecache pages
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004309] 0 pages in swap cache
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004310] Swap cache stats: add 0, delete 0, find 0/0
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004312] Free swap = 0kB
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.004313] Total swap = 0kB
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009822] 985072 pages RAM
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009824] 24998 pages reserved
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009825] 3789 pages shared
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009826] 953329 pages non-shared
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009828] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009837] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009840] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009844] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009847] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009850] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009853] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009857] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009860] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009864] [ 790] 101 790 63477 545 0 0 0 rsyslogd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009867] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009870] [ 831] 0 831 4226 41 0 0 0 atd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009874] [ 975] 0 975 6276 80 1 0 0 master
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009877] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009880] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009883] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009886] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009889] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009892] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009895] [15657] 0 15657 4777 60 1 0 0 cron
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009898] [ 654] 1012 654 70287 5737 1 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009901] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009904] [ 658] 1012 658 73718 10820 0 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009907] [ 678] 1012 678 70152 5559 1 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009911] [13468] 0 13468 89584 2540 1 0 0 php5-fpm
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009914] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009917] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009920] [25666] 1012 25666 68618 5632 0 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009923] [20652] 1012 20652 952535 889294 1 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009926] [ 7531] 1012 7531 68580 5604 0 0 0 php
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009929] Out of memory: Kill process 20652 (php) score 928 or sacrifice child
Jul 12 11:29:38 ip-10-61-174-46 kernel: [1155474.009939] Killed process 20652 (php) total-vm:3810140kB, anon-rss:3557176kB, file-rss:0kB
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116021] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116025] ntpd cpuset=/ mems_allowed=0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116028] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116030] Call Trace:
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116037] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116040] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116042] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116045] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116048] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116054] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116070] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116074] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116077] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116081] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116085] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116088] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116092] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116096] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116098] Mem-Info:
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116100] Node 0 DMA per-cpu:
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116102] CPU 0: hi: 0, btch: 1 usd: 0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116104] CPU 1: hi: 0, btch: 1 usd: 0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116106] Node 0 DMA32 per-cpu:
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116108] CPU 0: hi: 186, btch: 31 usd: 61
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116110] CPU 1: hi: 186, btch: 31 usd: 31
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116114] active_anon:932765 inactive_anon:1393 isolated_anon:0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116115] active_file:0 inactive_file:1 isolated_file:8
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116115] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116116] free:4036 slab_reclaimable:2736 slab_unreclaimable:2654
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116117] mapped:1364 shmem:1405 pagetables:3075 bounce:0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116120] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116128] lowmem_reserve[]: 0 3772 3772 3772
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116132] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3731060kB inactive_anon:5572kB active_file:0kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10944kB slab_unreclaimable:10616kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:85 all_unreclaimable? yes
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116140] lowmem_reserve[]: 0 0 0 0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116144] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116154] Node 0 DMA32: 1236*4kB 94*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116164] 1434 total pagecache pages
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116166] 0 pages in swap cache
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116167] Swap cache stats: add 0, delete 0, find 0/0
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116169] Free swap = 0kB
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.116170] Total swap = 0kB
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121630] 985072 pages RAM
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121631] 24998 pages reserved
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121632] 3773 pages shared
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121633] 953279 pages non-shared
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121635] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121643] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121645] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121648] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121650] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121652] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121654] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121657] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121659] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121662] [ 790] 101 790 63477 545 0 0 0 rsyslogd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121665] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121667] [ 831] 0 831 4226 41 0 0 0 atd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121669] [ 975] 0 975 6276 80 0 0 0 master
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121672] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121674] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121676] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121678] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121681] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121683] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121685] [15657] 0 15657 4777 60 0 0 0 cron
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121687] [ 654] 1012 654 70287 5737 0 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121689] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121691] [ 658] 1012 658 73718 10820 0 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121694] [ 678] 1012 678 70152 5559 1 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121696] [13468] 0 13468 89584 2533 1 0 0 php5-fpm
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121698] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121700] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121703] [25666] 1012 25666 68618 5633 0 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121705] [ 7595] 1012 7595 952140 888909 1 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121707] [ 5418] 108 5418 6792 69 1 0 0 pickup
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121710] [18897] 1012 18897 68580 5605 0 0 0 php
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121712] Out of memory: Kill process 7595 (php) score 927 or sacrifice child
Jul 12 17:29:37 ip-10-61-174-46 kernel: [1177073.121721] Killed process 7595 (php) total-vm:3808560kB, anon-rss:3555636kB, file-rss:0kB
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956202] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956206] php cpuset=/ mems_allowed=0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956209] Pid: 3519, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956210] Call Trace:
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956217] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956220] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956222] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956225] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956228] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956231] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956233] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956236] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956239] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956242] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956245] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956247] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956250] [<ffffffff811425b1>] ? find_vma+0x1/0x70
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956253] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956256] [<ffffffff811798c0>] ? vfs_write+0x110/0x180
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956259] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956261] Mem-Info:
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956262] Node 0 DMA per-cpu:
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956264] CPU 0: hi: 0, btch: 1 usd: 0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956265] CPU 1: hi: 0, btch: 1 usd: 0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956266] Node 0 DMA32 per-cpu:
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956268] CPU 0: hi: 186, btch: 31 usd: 35
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956269] CPU 1: hi: 186, btch: 31 usd: 36
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956272] active_anon:932814 inactive_anon:1393 isolated_anon:0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956273] active_file:21 inactive_file:10 isolated_file:0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956273] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956274] free:4044 slab_reclaimable:2714 slab_unreclaimable:2678
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956275] mapped:1360 shmem:1405 pagetables:3072 bounce:0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956276] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956283] lowmem_reserve[]: 0 3772 3772 3772
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956286] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731256kB inactive_anon:5572kB active_file:84kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:10856kB slab_unreclaimable:10712kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:239 all_unreclaimable? yes
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956292] lowmem_reserve[]: 0 0 0 0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956295] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956302] Node 0 DMA32: 1281*4kB 75*8kB 2*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956308] 1425 total pagecache pages
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956310] 0 pages in swap cache
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956311] Swap cache stats: add 0, delete 0, find 0/0
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956312] Free swap = 0kB
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.956313] Total swap = 0kB
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961780] 985072 pages RAM
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961783] 24998 pages reserved
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961784] 3774 pages shared
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961785] 953297 pages non-shared
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961787] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961796] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961799] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961802] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961805] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961809] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961812] [ 664] 0 664 1815 123 1 0 0 dhclient3
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961815] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961819] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961822] [ 790] 101 790 63477 569 0 0 0 rsyslogd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961825] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961828] [ 831] 0 831 4226 41 0 0 0 atd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961832] [ 975] 0 975 6276 80 1 0 0 master
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961835] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961838] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961841] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961844] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961847] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961850] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961854] [15657] 0 15657 4777 60 0 0 0 cron
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961857] [ 654] 1012 654 70287 5737 1 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961860] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961863] [ 658] 1012 658 73718 10820 0 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961866] [ 678] 1012 678 70152 5559 0 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961869] [13468] 0 13468 89584 2508 0 0 0 php5-fpm
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961872] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961875] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961878] [25666] 1012 25666 68618 5646 1 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961881] [18961] 1012 18961 952167 888945 0 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961885] [28937] 108 28937 6792 69 0 0 0 pickup
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961888] [ 3519] 1012 3519 68580 5604 0 0 0 php
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961891] Out of memory: Kill process 18961 (php) score 927 or sacrifice child
Jul 12 19:29:38 ip-10-61-174-46 kernel: [1184273.961901] Killed process 18961 (php) total-vm:3808668kB, anon-rss:3555780kB, file-rss:0kB
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116002] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116007] ntpd cpuset=/ mems_allowed=0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116010] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116012] Call Trace:
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116020] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116024] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116027] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116031] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116035] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116040] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116043] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116047] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116052] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116068] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116071] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116075] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116079] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116083] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116087] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116089] Mem-Info:
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116091] Node 0 DMA per-cpu:
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116094] CPU 0: hi: 0, btch: 1 usd: 0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116096] CPU 1: hi: 0, btch: 1 usd: 0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116097] Node 0 DMA32 per-cpu:
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116099] CPU 0: hi: 186, btch: 31 usd: 83
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116101] CPU 1: hi: 186, btch: 31 usd: 86
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116105] active_anon:931913 inactive_anon:1393 isolated_anon:0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116106] active_file:29 inactive_file:7 isolated_file:20
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116107] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116108] free:4048 slab_reclaimable:2861 slab_unreclaimable:2758
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116109] mapped:1427 shmem:1405 pagetables:3466 bounce:0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116111] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116119] lowmem_reserve[]: 0 3772 3772 3772
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116123] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727652kB inactive_anon:5572kB active_file:116kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5708kB shmem:5620kB slab_reclaimable:11444kB slab_unreclaimable:11032kB kernel_stack:840kB pagetables:13864kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:313 all_unreclaimable? yes
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116132] lowmem_reserve[]: 0 0 0 0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116135] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116145] Node 0 DMA32: 1459*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116155] 1458 total pagecache pages
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116157] 0 pages in swap cache
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116158] Swap cache stats: add 0, delete 0, find 0/0
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116160] Free swap = 0kB
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.116161] Total swap = 0kB
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121699] 985072 pages RAM
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121700] 24998 pages reserved
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121701] 3781 pages shared
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121702] 953140 pages non-shared
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121704] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121711] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121714] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121716] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121718] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121721] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121723] [ 664] 0 664 1815 123 1 0 0 dhclient3
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121725] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121728] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121731] [ 790] 101 790 63477 569 0 0 0 rsyslogd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121733] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121736] [ 831] 0 831 4226 41 0 0 0 atd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121738] [ 975] 0 975 6276 80 1 0 0 master
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121740] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121742] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121745] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121747] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121749] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121751] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121753] [15657] 0 15657 4777 60 0 0 0 cron
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121756] [ 654] 1012 654 70287 5737 1 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121758] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121760] [ 658] 1012 658 73718 10820 1 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121762] [ 678] 1012 678 70152 5559 0 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121764] [13468] 0 13468 89584 2493 0 0 0 php5-fpm
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121767] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121769] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121771] [25666] 1012 25666 68618 5657 0 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121774] [23848] 1012 23848 943905 880691 0 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121776] [10738] 108 10738 6792 70 0 0 0 pickup
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121778] [19236] 108 19236 6818 73 0 0 0 cleanup
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121781] [19237] 108 19237 6795 71 0 0 0 trivial-rewrite
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121783] [19238] 108 19238 11107 159 1 0 0 smtp
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121785] [19245] 108 19245 11107 158 0 0 0 smtp
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121787] [19249] 108 19249 11107 158 1 0 0 smtp
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121789] [19494] 1012 19494 71025 8073 1 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121791] [19591] 1012 19591 1099 25 1 0 0 sh
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121793] [19592] 1012 19592 1099 24 1 0 0 sh
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121795] [19593] 1012 19593 64043 1403 0 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121798] [19594] 1012 19594 64040 1401 1 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121800] [19595] 1012 19595 1099 25 1 0 0 sh
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121802] [19596] 1012 19596 63985 1367 1 0 0 php
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121804] Out of memory: Kill process 23848 (php) score 919 or sacrifice child
Jul 12 23:29:40 ip-10-61-174-46 kernel: [1198676.121814] Killed process 23848 (php) total-vm:3775620kB, anon-rss:3522764kB, file-rss:0kB
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868215] git invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868218] git cpuset=/ mems_allowed=0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868221] Pid: 31369, comm: git Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868223] Call Trace:
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868230] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868232] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868234] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868237] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868240] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868244] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868246] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868249] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868253] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868256] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868258] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868260] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868264] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868266] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868269] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868271] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868273] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868276] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868278] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868281] [<ffffffff81184a25>] ? putname+0x35/0x50
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868284] [<ffffffff81091d20>] ? release_tgcred.isra.4+0x30/0x30
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868287] [<ffffffff8100ad8f>] ? xen_restore_fl_direct_reloc+0x4/0x4
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868290] [<ffffffff810e090d>] ? __call_rcu+0x9d/0x170
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868293] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868295] [<ffffffff810e0a15>] ? call_rcu_sched+0x15/0x20
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868297] [<ffffffff81091dff>] ? __put_cred+0x3f/0x50
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868300] [<ffffffff811783d4>] ? sys_faccessat+0x114/0x1e0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868302] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868304] Mem-Info:
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868305] Node 0 DMA per-cpu:
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868307] CPU 0: hi: 0, btch: 1 usd: 0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868308] CPU 1: hi: 0, btch: 1 usd: 0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868309] Node 0 DMA32 per-cpu:
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868311] CPU 0: hi: 186, btch: 31 usd: 30
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868312] CPU 1: hi: 186, btch: 31 usd: 27
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868315] active_anon:931953 inactive_anon:1393 isolated_anon:0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868316] active_file:0 inactive_file:21 isolated_file:0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868316] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868317] free:4029 slab_reclaimable:3105 slab_unreclaimable:2754
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868318] mapped:1359 shmem:1405 pagetables:3253 bounce:0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868319] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868326] lowmem_reserve[]: 0 3772 3772 3772
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868329] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3727812kB inactive_anon:5572kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:12420kB slab_unreclaimable:11016kB kernel_stack:808kB pagetables:13012kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868335] lowmem_reserve[]: 0 0 0 0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868338] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868344] Node 0 DMA32: 1419*4kB 3*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7780kB
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868351] 1457 total pagecache pages
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868352] 0 pages in swap cache
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868353] Swap cache stats: add 0, delete 0, find 0/0
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868355] Free swap = 0kB
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.868356] Total swap = 0kB
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873783] 985072 pages RAM
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873785] 24998 pages reserved
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873787] 3805 pages shared
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873788] 953284 pages non-shared
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873789] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873798] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873802] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873805] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873808] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873811] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873814] [ 664] 0 664 1815 123 1 0 0 dhclient3
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873818] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873821] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873825] [ 790] 101 790 63477 569 0 0 0 rsyslogd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873828] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873831] [ 831] 0 831 4226 41 0 0 0 atd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873834] [ 975] 0 975 6276 80 0 0 0 master
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873837] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873840] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873844] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873847] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873850] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873853] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873856] [15657] 0 15657 4777 60 0 0 0 cron
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873859] [ 654] 1012 654 70287 5738 1 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873863] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873866] [ 658] 1012 658 73782 10849 1 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873869] [ 678] 1012 678 70152 5563 1 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873872] [13468] 0 13468 89584 2510 0 0 0 php5-fpm
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873875] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873878] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873881] [22638] 1012 22638 68618 5647 0 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873884] [11638] 1012 11638 950168 886834 0 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873887] [20820] 108 20820 6792 69 0 0 0 pickup
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873891] [26906] 108 26906 6818 73 0 0 0 cleanup
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873894] [26907] 108 26907 6795 71 0 0 0 trivial-rewrite
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873897] [26908] 108 26908 11107 161 1 0 0 smtp
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873900] [26916] 108 26916 11107 159 0 0 0 smtp
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873903] [28599] 108 28599 6792 70 0 0 0 scache
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873906] [28614] 108 28614 11108 160 1 0 0 smtp
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873910] [28623] 108 28623 11108 159 0 0 0 smtp
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873913] [31365] 1012 31365 1099 24 1 0 0 sh
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873916] [31366] 1012 31366 68825 5847 1 0 0 php
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873919] [31368] 1012 31368 1099 26 1 0 0 sh
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873922] [31369] 1012 31369 5065 64 1 0 0 git
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873924] Out of memory: Kill process 11638 (php) score 925 or sacrifice child
Jul 13 03:29:43 ip-10-61-174-46 kernel: [1213078.873935] Killed process 11638 (php) total-vm:3800672kB, anon-rss:3547336kB, file-rss:0kB
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268130] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268135] php cpuset=/ mems_allowed=0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268138] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268140] Call Trace:
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268158] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268162] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268166] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268169] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268173] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268177] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268181] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268184] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268187] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268191] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268194] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268198] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268202] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268205] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268209] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268212] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268214] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268217] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268219] Mem-Info:
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268220] Node 0 DMA per-cpu:
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268222] CPU 0: hi: 0, btch: 1 usd: 0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268223] CPU 1: hi: 0, btch: 1 usd: 0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268224] Node 0 DMA32 per-cpu:
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268226] CPU 0: hi: 186, btch: 31 usd: 0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268227] CPU 1: hi: 186, btch: 31 usd: 0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268230] active_anon:932800 inactive_anon:1393 isolated_anon:0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268230] active_file:15 inactive_file:24 isolated_file:0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268231] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268232] free:4048 slab_reclaimable:2595 slab_unreclaimable:2719
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268233] mapped:1386 shmem:1405 pagetables:3138 bounce:0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268241] lowmem_reserve[]: 0 3772 3772 3772
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268244] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731200kB inactive_anon:5572kB active_file:60kB inactive_file:96kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:10380kB slab_unreclaimable:10876kB kernel_stack:736kB pagetables:12552kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6338 all_unreclaimable? yes
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268250] lowmem_reserve[]: 0 0 0 0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268260] Node 0 DMA32: 1456*4kB 10*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7968kB
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268266] 1445 total pagecache pages
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268267] 0 pages in swap cache
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268269] Swap cache stats: add 0, delete 0, find 0/0
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268270] Free swap = 0kB
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.268271] Total swap = 0kB
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273575] 985072 pages RAM
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273578] 24998 pages reserved
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273579] 3802 pages shared
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273580] 953306 pages non-shared
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273581] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273591] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273594] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273597] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273600] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273603] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273607] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273610] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273613] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273617] [ 790] 101 790 63477 569 0 0 0 rsyslogd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273621] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273624] [ 831] 0 831 4226 41 0 0 0 atd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273627] [ 975] 0 975 6276 80 0 0 0 master
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273631] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273634] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273637] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273640] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273643] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273646] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273649] [15657] 0 15657 4777 60 0 0 0 cron
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273652] [ 654] 1012 654 70287 5738 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273655] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273658] [ 658] 1012 658 73782 10849 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273661] [ 678] 1012 678 70152 5562 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273664] [13468] 0 13468 89584 2535 1 0 0 php5-fpm
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273668] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273671] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273674] [22638] 1012 22638 68618 5647 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273677] [24070] 108 24070 6792 70 1 0 0 pickup
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273680] [10322] 1012 10322 946534 883279 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273685] [16803] 1012 16803 68580 5605 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273688] [16820] 1012 16820 68580 5604 1 0 0 php
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273691] Out of memory: Kill process 10322 (php) score 921 or sacrifice child
Jul 13 05:29:43 ip-10-61-174-46 kernel: [1220279.273702] Killed process 10322 (php) total-vm:3786136kB, anon-rss:3533028kB, file-rss:88kB
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908326] rsyslogd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908331] rsyslogd cpuset=/ mems_allowed=0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908334] Pid: 790, comm: rsyslogd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908336] Call Trace:
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908344] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908347] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908350] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908354] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908361] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908372] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908378] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908381] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908385] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908388] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908391] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908394] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908398] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908403] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908406] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908409] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908414] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908418] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908421] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908424] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908427] Mem-Info:
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908428] Node 0 DMA per-cpu:
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908431] CPU 0: hi: 0, btch: 1 usd: 0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908433] CPU 1: hi: 0, btch: 1 usd: 0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908436] Node 0 DMA32 per-cpu:
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908438] CPU 0: hi: 186, btch: 31 usd: 0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908440] CPU 1: hi: 186, btch: 31 usd: 4
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908444] active_anon:932453 inactive_anon:1393 isolated_anon:0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908445] active_file:0 inactive_file:17 isolated_file:5
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908446] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908447] free:4048 slab_reclaimable:2953 slab_unreclaimable:2690
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908448] mapped:1366 shmem:1405 pagetables:3170 bounce:0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908450] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908458] lowmem_reserve[]: 0 3772 3772 3772
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908462] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729812kB inactive_anon:5572kB active_file:0kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):20kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:11812kB slab_unreclaimable:10760kB kernel_stack:744kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:111 all_unreclaimable? yes
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908471] lowmem_reserve[]: 0 0 0 0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908474] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908484] Node 0 DMA32: 1448*4kB 13*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7960kB
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908493] 1443 total pagecache pages
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908494] 0 pages in swap cache
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908496] Swap cache stats: add 0, delete 0, find 0/0
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908498] Free swap = 0kB
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.908499] Total swap = 0kB
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913975] 985072 pages RAM
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913977] 24998 pages reserved
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913978] 3777 pages shared
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913980] 953328 pages non-shared
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913982] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913990] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913994] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913996] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.913999] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914002] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914005] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914008] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914012] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914015] [ 790] 101 790 63477 583 0 0 0 rsyslogd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914018] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914021] [ 831] 0 831 4226 41 0 0 0 atd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914024] [ 975] 0 975 6276 80 0 0 0 master
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914027] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914030] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914033] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914036] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914039] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914042] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914046] [15657] 0 15657 4777 60 1 0 0 cron
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914049] [ 654] 1012 654 70287 5737 1 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914052] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914055] [ 658] 1012 658 73782 10849 0 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914058] [ 678] 1012 678 70152 5562 1 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914061] [13468] 0 13468 89584 2504 1 0 0 php5-fpm
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914064] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914067] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914070] [22638] 1012 22638 68682 5692 1 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914073] [23445] 1012 23445 945905 882582 1 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914076] [31951] 108 31951 6792 69 1 0 0 pickup
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914079] [ 8726] 1012 8726 68580 5603 1 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914082] [ 8728] 1012 8728 1099 24 0 0 0 sh
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914085] [ 8729] 1012 8729 68825 5871 1 0 0 php
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914088] Out of memory: Kill process 23445 (php) score 921 or sacrifice child
Jul 13 15:29:49 ip-10-61-174-46 kernel: [1256284.914099] Killed process 23445 (php) total-vm:3783620kB, anon-rss:3530328kB, file-rss:0kB
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942840] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942844] php cpuset=/ mems_allowed=0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942846] Pid: 18876, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942848] Call Trace:
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942855] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942857] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942860] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942863] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942866] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942870] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942873] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942875] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942878] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942881] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942884] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942886] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942888] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942891] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942895] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942897] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942900] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942903] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942906] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942908] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942911] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942913] Mem-Info:
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942914] Node 0 DMA per-cpu:
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942916] CPU 0: hi: 0, btch: 1 usd: 0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942918] CPU 1: hi: 0, btch: 1 usd: 0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942919] Node 0 DMA32 per-cpu:
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942920] CPU 0: hi: 186, btch: 31 usd: 50
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942922] CPU 1: hi: 186, btch: 31 usd: 37
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942924] active_anon:932466 inactive_anon:1393 isolated_anon:0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942925] active_file:43 inactive_file:6 isolated_file:12
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942926] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942926] free:4034 slab_reclaimable:2934 slab_unreclaimable:2628
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942927] mapped:1342 shmem:1405 pagetables:3160 bounce:0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942929] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942935] lowmem_reserve[]: 0 3772 3772 3772
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942938] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3729864kB inactive_anon:5572kB active_file:172kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5368kB shmem:5620kB slab_reclaimable:11736kB slab_unreclaimable:10512kB kernel_stack:760kB pagetables:12640kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:381 all_unreclaimable? yes
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942945] lowmem_reserve[]: 0 0 0 0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942947] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942954] Node 0 DMA32: 1428*4kB 3*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942961] 1418 total pagecache pages
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942963] 0 pages in swap cache
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942964] Swap cache stats: add 0, delete 0, find 0/0
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942965] Free swap = 0kB
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.942966] Total swap = 0kB
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948128] 985072 pages RAM
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948130] 24998 pages reserved
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948131] 3775 pages shared
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948133] 953285 pages non-shared
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948134] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948144] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948148] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948151] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948154] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948157] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948161] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948164] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948167] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948171] [ 790] 101 790 63477 583 1 0 0 rsyslogd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948174] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948178] [ 831] 0 831 4226 41 0 0 0 atd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948181] [ 975] 0 975 6276 80 0 0 0 master
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948184] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948187] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948190] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948193] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948196] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948199] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948203] [15657] 0 15657 4777 60 1 0 0 cron
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948206] [ 654] 1012 654 70287 5737 1 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948209] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948212] [ 658] 1012 658 76918 14029 1 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948215] [ 678] 1012 678 70152 5562 0 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948218] [13468] 0 13468 89584 2508 0 0 0 php5-fpm
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948221] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948224] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948227] [22638] 1012 22638 68682 5692 1 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948230] [16072] 1012 16072 948043 884849 0 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948234] [ 4842] 108 4842 6792 69 0 0 0 pickup
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948237] [18805] 108 18805 6818 73 0 0 0 cleanup
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948240] [18806] 108 18806 6795 70 1 0 0 trivial-rewrite
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948243] [18807] 108 18807 11107 158 0 0 0 smtp
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948246] [18811] 108 18811 11107 158 0 0 0 smtp
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948249] [18876] 1012 18876 68580 5604 1 0 0 php
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948252] Out of memory: Kill process 16072 (php) score 923 or sacrifice child
Jul 13 19:29:50 ip-10-61-174-46 kernel: [1270685.948264] Killed process 16072 (php) total-vm:3792172kB, anon-rss:3539396kB, file-rss:0kB
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249004] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249009] php cpuset=/ mems_allowed=0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249012] Pid: 13997, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249014] Call Trace:
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249021] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249023] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249025] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249028] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249032] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249035] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249037] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249040] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249043] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249046] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249048] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249051] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249054] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249058] [<ffffffff8117e6ea>] ? sys_newlstat+0x2a/0x40
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249061] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249062] Mem-Info:
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249064] Node 0 DMA per-cpu:
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249065] CPU 0: hi: 0, btch: 1 usd: 0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249067] CPU 1: hi: 0, btch: 1 usd: 0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249068] Node 0 DMA32 per-cpu:
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249070] CPU 0: hi: 186, btch: 31 usd: 30
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249071] CPU 1: hi: 186, btch: 31 usd: 157
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249073] active_anon:932573 inactive_anon:1393 isolated_anon:0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249074] active_file:12 inactive_file:13 isolated_file:0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249075] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249075] free:4036 slab_reclaimable:2701 slab_unreclaimable:2605
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249076] mapped:1362 shmem:1405 pagetables:3228 bounce:0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249078] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249084] lowmem_reserve[]: 0 3772 3772 3772
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249087] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3730292kB inactive_anon:5572kB active_file:48kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:10804kB slab_unreclaimable:10420kB kernel_stack:776kB pagetables:12912kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:329 all_unreclaimable? yes
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249094] lowmem_reserve[]: 0 0 0 0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249096] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249103] Node 0 DMA32: 1064*4kB 188*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249110] 1461 total pagecache pages
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249111] 0 pages in swap cache
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249112] Swap cache stats: add 0, delete 0, find 0/0
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249113] Free swap = 0kB
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.249114] Total swap = 0kB
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254324] 985072 pages RAM
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254326] 24998 pages reserved
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254327] 3793 pages shared
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254328] 953169 pages non-shared
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254330] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254339] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254343] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254346] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254349] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254352] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254356] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254359] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254362] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254366] [ 790] 101 790 63477 583 0 0 0 rsyslogd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254369] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254372] [ 831] 0 831 4226 41 0 0 0 atd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254376] [ 975] 0 975 6276 80 0 0 0 master
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254379] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254382] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254385] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254388] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254391] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254394] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254397] [15657] 0 15657 4777 60 0 0 0 cron
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254401] [ 654] 1012 654 70287 5738 0 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254404] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254407] [ 658] 1012 658 76918 14029 0 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254410] [ 678] 1012 678 70152 5562 0 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254413] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254416] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254419] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254422] [22638] 1012 22638 68682 5692 1 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254426] [19154] 1012 19154 950704 887465 1 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254428] [24159] 108 24159 6792 69 0 0 0 pickup
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254432] [13866] 108 13866 6818 73 0 0 0 cleanup
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254435] [13867] 108 13867 6795 70 0 0 0 trivial-rewrite
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254438] [13868] 108 13868 11107 157 1 0 0 smtp
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254442] [13994] 1012 13994 1099 24 0 0 0 sh
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254444] [13995] 1012 13995 1099 26 0 0 0 sh
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254448] [13996] 1012 13996 64213 1627 1 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254450] [13997] 1012 13997 64213 1581 0 0 0 php
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254453] Out of memory: Kill process 19154 (php) score 926 or sacrifice child
Jul 13 20:29:32 ip-10-61-174-46 kernel: [1274268.254465] Killed process 19154 (php) total-vm:3802816kB, anon-rss:3549860kB, file-rss:0kB
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164121] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164125] php cpuset=/ mems_allowed=0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164129] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164131] Call Trace:
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164139] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164145] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164148] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164153] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164155] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164159] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164161] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164164] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164167] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164170] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164172] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164175] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164177] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164180] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164183] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164186] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164190] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164193] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164196] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164198] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164200] Mem-Info:
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164201] Node 0 DMA per-cpu:
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164203] CPU 0: hi: 0, btch: 1 usd: 0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164204] CPU 1: hi: 0, btch: 1 usd: 0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164205] Node 0 DMA32 per-cpu:
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164207] CPU 0: hi: 186, btch: 31 usd: 0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164208] CPU 1: hi: 186, btch: 31 usd: 0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164211] active_anon:932667 inactive_anon:1393 isolated_anon:0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164212] active_file:0 inactive_file:0 isolated_file:0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164212] unevictable:0 dirty:10 writeback:0 unstable:0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164213] free:4048 slab_reclaimable:2816 slab_unreclaimable:2700
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164214] mapped:1375 shmem:1405 pagetables:3051 bounce:0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164215] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164222] lowmem_reserve[]: 0 3772 3772 3772
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164225] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730668kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:40kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:11264kB slab_unreclaimable:10800kB kernel_stack:728kB pagetables:12204kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164231] lowmem_reserve[]: 0 0 0 0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164234] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164241] Node 0 DMA32: 1469*4kB 2*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164247] 1432 total pagecache pages
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164248] 0 pages in swap cache
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164249] Swap cache stats: add 0, delete 0, find 0/0
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164251] Free swap = 0kB
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.164251] Total swap = 0kB
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169387] 985072 pages RAM
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169390] 24998 pages reserved
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169391] 3800 pages shared
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169392] 953316 pages non-shared
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169394] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169403] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169406] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169409] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169412] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169416] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169419] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169422] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169425] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169429] [ 790] 101 790 63477 583 1 0 0 rsyslogd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169433] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169436] [ 831] 0 831 4226 41 0 0 0 atd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169439] [ 975] 0 975 6276 80 0 0 0 master
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169442] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169445] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169448] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169451] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169455] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169458] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169461] [15657] 0 15657 4777 60 0 0 0 cron
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169464] [ 654] 1012 654 70287 5738 1 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169467] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169470] [ 658] 1012 658 135801 72207 1 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169473] [ 678] 1012 678 70152 5562 1 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169476] [13468] 0 13468 89584 2524 0 0 0 php5-fpm
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169479] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169482] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169485] [22638] 1012 22638 68682 5692 1 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169489] [27214] 1012 27214 890524 827357 1 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169492] [ 2211] 108 2211 6792 69 0 0 0 pickup
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169495] [ 6395] 1012 6395 68580 5605 1 0 0 php
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169498] Out of memory: Kill process 27214 (php) score 863 or sacrifice child
Jul 13 23:29:52 ip-10-61-174-46 kernel: [1285088.169509] Killed process 27214 (php) total-vm:3562096kB, anon-rss:3309428kB, file-rss:0kB
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805400] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805405] php cpuset=/ mems_allowed=0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805408] Pid: 6731, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805410] Call Trace:
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805418] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805421] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805424] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805428] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805434] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805438] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805442] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805445] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805449] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805453] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805456] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805460] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805463] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805467] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805472] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805479] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805483] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805486] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805489] Mem-Info:
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805490] Node 0 DMA per-cpu:
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805492] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805494] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805495] Node 0 DMA32 per-cpu:
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805498] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805499] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805503] active_anon:932829 inactive_anon:1393 isolated_anon:0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805504] active_file:2 inactive_file:0 isolated_file:0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805505] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805506] free:4048 slab_reclaimable:2746 slab_unreclaimable:2648
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805507] mapped:1374 shmem:1405 pagetables:3071 bounce:0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805509] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805517] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805521] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731316kB inactive_anon:5572kB active_file:8kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5620kB slab_reclaimable:10984kB slab_unreclaimable:10592kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24 all_unreclaimable? yes
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805529] lowmem_reserve[]: 0 0 0 0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805533] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805542] Node 0 DMA32: 1451*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7900kB
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805552] 1419 total pagecache pages
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805553] 0 pages in swap cache
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805555] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805557] Free swap = 0kB
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.805558] Total swap = 0kB
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810902] 985072 pages RAM
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810905] 24998 pages reserved
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810906] 3788 pages shared
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810907] 953326 pages non-shared
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810909] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810918] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810921] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810924] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810927] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810930] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810934] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810937] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810941] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810944] [ 790] 101 790 63477 583 0 0 0 rsyslogd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810948] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810951] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810954] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810957] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810960] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810963] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810967] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810970] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810973] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810976] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810980] [ 654] 1012 654 70287 5738 0 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810982] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810985] [ 658] 1012 658 135801 72207 0 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810988] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810992] [13468] 0 13468 89584 2509 0 0 0 php5-fpm
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810995] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.810998] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811001] [ 6731] 1012 6731 890805 827560 1 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811004] [ 2882] 108 2882 6792 70 1 0 0 pickup
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811007] [ 8112] 1012 8112 68610 5635 1 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811010] [ 9092] 1012 9092 68580 5605 1 0 0 php
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811013] Out of memory: Kill process 6731 (php) score 863 or sacrifice child
Jul 14 00:29:32 ip-10-61-174-46 kernel: [1288667.811023] Killed process 6731 (php) total-vm:3563220kB, anon-rss:3310240kB, file-rss:0kB
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212160] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212164] php cpuset=/ mems_allowed=0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212167] Pid: 10012, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212169] Call Trace:
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212175] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212178] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212180] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212183] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212186] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212190] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212192] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212195] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212200] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212203] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212205] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212208] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212211] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212214] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212216] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212218] Mem-Info:
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212219] Node 0 DMA per-cpu:
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212221] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212223] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212224] Node 0 DMA32 per-cpu:
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212226] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212227] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212229] active_anon:932397 inactive_anon:1393 isolated_anon:0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212230] active_file:0 inactive_file:0 isolated_file:0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212231] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212231] free:4048 slab_reclaimable:2921 slab_unreclaimable:2714
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212232] mapped:1384 shmem:1405 pagetables:3211 bounce:0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212240] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212243] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729588kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5536kB shmem:5620kB slab_reclaimable:11684kB slab_unreclaimable:10856kB kernel_stack:784kB pagetables:12844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212250] lowmem_reserve[]: 0 0 0 0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212252] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212259] Node 0 DMA32: 1462*4kB 7*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212265] 1437 total pagecache pages
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212267] 0 pages in swap cache
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212268] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212269] Free swap = 0kB
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.212270] Total swap = 0kB
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217372] 985072 pages RAM
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217375] 24998 pages reserved
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217376] 3810 pages shared
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217377] 953296 pages non-shared
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217379] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217388] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217392] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217395] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217398] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217401] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217404] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217408] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217411] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217415] [ 790] 101 790 63477 599 0 0 0 rsyslogd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217418] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217421] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217424] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217427] [ 983] 108 983 6834 106 0 0 0 qmgr
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217430] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217434] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217437] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217440] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217443] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217446] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217449] [ 654] 1012 654 70287 5738 0 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217452] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217455] [ 658] 1012 658 135801 72207 0 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217458] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217462] [13468] 0 13468 89584 2494 0 0 0 php5-fpm
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217465] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217468] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217471] [ 8112] 1012 8112 68610 5642 0 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217474] [10012] 1012 10012 883193 820220 0 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217477] [ 9949] 108 9949 6792 69 0 0 0 scache
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217480] [10950] 108 10950 6818 72 0 0 0 cleanup
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217483] [10951] 108 10951 6795 71 0 0 0 trivial-rewrite
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217486] [10952] 108 10952 11107 161 0 0 0 smtp
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217490] [10956] 108 10956 11107 161 0 0 0 smtp
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217493] [10960] 108 10960 11107 160 0 0 0 smtp
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217496] [11402] 108 11402 11107 160 0 0 0 smtp
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217499] [11418] 108 11418 11108 159 0 0 0 smtp
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217502] [11732] 1012 11732 74525 11582 0 0 0 php
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217505] Out of memory: Kill process 10012 (php) score 856 or sacrifice child
Jul 14 01:33:19 ip-10-61-174-46 kernel: [1292495.217516] Killed process 10012 (php) total-vm:3532772kB, anon-rss:3280792kB, file-rss:88kB
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500118] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500122] php cpuset=/ mems_allowed=0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500124] Pid: 4382, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500126] Call Trace:
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500133] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500135] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500137] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500140] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500143] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500147] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500149] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500152] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500155] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500157] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500161] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500163] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500166] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500169] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500172] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500175] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500177] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500179] Mem-Info:
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500180] Node 0 DMA per-cpu:
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500182] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500184] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500185] Node 0 DMA32 per-cpu:
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500186] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500187] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500190] active_anon:931818 inactive_anon:1393 isolated_anon:0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500191] active_file:0 inactive_file:0 isolated_file:0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500191] unevictable:0 dirty:11 writeback:0 unstable:0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500192] free:4048 slab_reclaimable:3320 slab_unreclaimable:2783
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500193] mapped:1395 shmem:1405 pagetables:3291 bounce:0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500194] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500201] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500204] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727272kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:44kB writeback:0kB mapped:5580kB shmem:5620kB slab_reclaimable:13280kB slab_unreclaimable:11132kB kernel_stack:808kB pagetables:13164kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1 all_unreclaimable? yes
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500211] lowmem_reserve[]: 0 0 0 0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500213] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500220] Node 0 DMA32: 1441*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500226] 1443 total pagecache pages
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500228] 0 pages in swap cache
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500229] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500230] Free swap = 0kB
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.500231] Total swap = 0kB
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505412] 985072 pages RAM
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505414] 24998 pages reserved
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505415] 4084 pages shared
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505416] 953309 pages non-shared
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505418] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505427] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505430] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505433] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505436] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505439] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505442] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505445] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505449] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505452] [ 790] 101 790 63477 599 0 0 0 rsyslogd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505455] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505457] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505460] [ 975] 0 975 6276 88 1 0 0 master
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505463] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505466] [ 1024] 0 1024 3309 39 0 0 0 mdadm
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505468] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505471] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505474] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505476] [ 1242] 107 1242 9445 166 1 0 0 ntpd
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505479] [15657] 0 15657 4777 61 0 0 0 cron
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505482] [ 654] 1012 654 70287 5768 1 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505484] [ 657] 1012 657 69242 6282 1 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505487] [ 658] 1012 658 135033 71460 1 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505490] [ 678] 1012 678 70152 5567 1 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505494] [13468] 0 13468 89584 2524 1 0 0 php5-fpm
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505497] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505501] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505504] [ 8112] 1012 8112 68610 5677 1 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505507] [ 4382] 1012 4382 889105 825985 0 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505511] [ 2671] 108 2671 6792 70 0 0 0 pickup
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505514] [ 5855] 108 5855 6818 73 0 0 0 cleanup
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505517] [ 5856] 108 5856 6795 70 1 0 0 trivial-rewrite
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505519] [ 5857] 108 5857 11107 167 1 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505522] [ 5861] 108 5861 11107 169 1 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505526] [ 7595] 108 7595 11107 166 0 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505528] [ 7608] 108 7608 6792 69 0 0 0 scache
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505531] [ 7624] 108 7624 11108 167 0 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505534] [ 7630] 108 7630 11108 166 0 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505537] [ 7637] 108 7637 11108 168 0 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505541] [ 7641] 108 7641 11108 167 1 0 0 smtp
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505544] [ 7825] 1012 7825 68580 5629 1 0 0 php
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505548] Out of memory: Kill process 4382 (php) score 862 or sacrifice child
Jul 14 03:29:52 ip-10-61-174-46 kernel: [1299488.505559] Killed process 4382 (php) total-vm:3556420kB, anon-rss:3303804kB, file-rss:124kB
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809443] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809447] php5-fpm cpuset=/ mems_allowed=0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809450] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809452] Call Trace:
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809459] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809461] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809464] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809466] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809470] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809473] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809476] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809478] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809482] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809485] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809488] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809490] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809493] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809495] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809498] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809501] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809504] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809508] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809511] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809514] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809516] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809520] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809522] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809524] Mem-Info:
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809525] Node 0 DMA per-cpu:
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809527] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809528] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809529] Node 0 DMA32 per-cpu:
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809531] CPU 0: hi: 186, btch: 31 usd: 33
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809532] CPU 1: hi: 186, btch: 31 usd: 27
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809535] active_anon:932826 inactive_anon:1393 isolated_anon:0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809536] active_file:6 inactive_file:0 isolated_file:0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809536] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809537] free:4040 slab_reclaimable:2425 slab_unreclaimable:2633
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809538] mapped:1375 shmem:1405 pagetables:3298 bounce:0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809539] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809546] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809549] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3731304kB inactive_anon:5572kB active_file:24kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:9700kB slab_unreclaimable:10532kB kernel_stack:776kB pagetables:13192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:275 all_unreclaimable? yes
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809555] lowmem_reserve[]: 0 0 0 0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809558] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809564] Node 0 DMA32: 1008*4kB 218*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809571] 1410 total pagecache pages
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809572] 0 pages in swap cache
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809573] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809574] Free swap = 0kB
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.809575] Total swap = 0kB
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814948] 985072 pages RAM
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814950] 24998 pages reserved
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814952] 3775 pages shared
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814953] 953304 pages non-shared
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814955] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814963] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814966] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814970] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814973] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814976] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814979] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814982] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814986] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814989] [ 790] 101 790 63477 599 1 0 0 rsyslogd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814993] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814996] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.814999] [ 975] 0 975 6276 79 1 0 0 master
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815002] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815005] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815008] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815011] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815014] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815018] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815021] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815024] [ 654] 1012 654 70287 5743 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815027] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815030] [ 658] 1012 658 135033 71439 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815033] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815036] [13468] 0 13468 89584 2528 0 0 0 php5-fpm
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815039] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815042] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815045] [ 8112] 1012 8112 68610 5652 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815049] [ 2671] 108 2671 6792 76 0 0 0 pickup
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815052] [ 7608] 108 7608 6792 69 1 0 0 scache
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815055] [ 8743] 1012 8743 884611 821520 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815058] [ 9477] 1012 9477 68580 5604 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815062] [ 9558] 1012 9558 1099 25 1 0 0 sh
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815065] [ 9559] 1012 9559 68825 5871 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815068] [ 9565] 1012 9565 1099 26 1 0 0 sh
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815071] [ 9566] 1012 9566 14040 63 1 0 0 git
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815074] [ 9567] 1012 9567 43910 713 1 0 0 php
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815076] Out of memory: Kill process 8743 (php) score 857 or sacrifice child
Jul 14 03:33:21 ip-10-61-174-46 kernel: [1299696.815088] Killed process 8743 (php) total-vm:3538444kB, anon-rss:3286080kB, file-rss:0kB
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024285] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024289] php cpuset=/ mems_allowed=0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024292] Pid: 12196, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024294] Call Trace:
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024301] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024303] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024305] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024308] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024312] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024315] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024317] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024320] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024325] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024327] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024331] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024333] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024335] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024338] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024341] [<ffffffff8106bc3c>] ? do_wait+0x12c/0x260
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024344] [<ffffffff8106cf4b>] ? sys_wait4+0xab/0xf0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024347] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024349] Mem-Info:
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024350] Node 0 DMA per-cpu:
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024352] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024353] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024354] Node 0 DMA32 per-cpu:
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024356] CPU 0: hi: 186, btch: 31 usd: 30
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024357] CPU 1: hi: 186, btch: 31 usd: 57
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024360] active_anon:932473 inactive_anon:1393 isolated_anon:0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024360] active_file:1 inactive_file:0 isolated_file:0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024361] unevictable:0 dirty:0 writeback:11 unstable:0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024362] free:4018 slab_reclaimable:2750 slab_unreclaimable:2649
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024362] mapped:1356 shmem:1405 pagetables:3311 bounce:0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024364] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024370] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024373] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3729892kB inactive_anon:5572kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:44kB mapped:5424kB shmem:5620kB slab_reclaimable:11000kB slab_unreclaimable:10596kB kernel_stack:784kB pagetables:13244kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:105 all_unreclaimable? yes
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024380] lowmem_reserve[]: 0 0 0 0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024383] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024389] Node 0 DMA32: 1375*4kB 26*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024396] 1442 total pagecache pages
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024397] 0 pages in swap cache
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024398] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024399] Free swap = 0kB
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.024400] Total swap = 0kB
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029662] 985072 pages RAM
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029664] 24998 pages reserved
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029666] 3790 pages shared
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029667] 953278 pages non-shared
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029669] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029677] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029681] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029684] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029687] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029690] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029694] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029697] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029700] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029704] [ 790] 101 790 63477 599 0 0 0 rsyslogd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029707] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029711] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029714] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029717] [ 983] 108 983 6834 106 1 0 0 qmgr
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029720] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029723] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029726] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029729] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029732] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029736] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029739] [ 654] 1012 654 70287 5743 1 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029742] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029745] [ 658] 1012 658 135033 71439 0 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029748] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029751] [13468] 0 13468 89584 2512 0 0 0 php5-fpm
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029754] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029757] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029760] [ 8112] 1012 8112 68610 5653 1 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029763] [13068] 1012 13068 879395 816196 1 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029767] [ 7952] 108 7952 6792 70 1 0 0 pickup
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029770] [12163] 1012 12163 68580 5604 0 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029773] [12195] 1012 12195 1099 25 0 0 0 sh
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029776] [12196] 1012 12196 68825 5870 0 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029779] [12202] 1012 12202 1099 25 0 0 0 sh
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029782] [12203] 1012 12203 4984 64 1 0 0 git
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029785] [12204] 1012 12204 68499 5551 0 0 0 php
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029788] [12206] 1012 12206 1099 26 0 0 0 sh
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029791] [12207] 1012 12207 10901 175 0 0 0 ssh
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029796] Out of memory: Kill process 13068 (php) score 851 or sacrifice child
Jul 14 05:29:52 ip-10-61-174-46 kernel: [1306688.029807] Killed process 13068 (php) total-vm:3517580kB, anon-rss:3264784kB, file-rss:0kB
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272143] php cpuset=/ mems_allowed=0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272147] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272149] Call Trace:
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272156] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272159] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272175] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272186] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272189] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272193] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272197] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272203] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272206] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272210] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272213] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272215] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272217] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272221] Mem-Info:
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272222] Node 0 DMA per-cpu:
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272224] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272225] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272226] Node 0 DMA32 per-cpu:
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272228] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272229] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272232] active_anon:932347 inactive_anon:1393 isolated_anon:0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272233] active_file:0 inactive_file:11 isolated_file:0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272233] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272234] free:4048 slab_reclaimable:3045 slab_unreclaimable:2706
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272235] mapped:1354 shmem:1405 pagetables:3131 bounce:0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272236] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272242] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272245] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729388kB inactive_anon:5572kB active_file:0kB inactive_file:44kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:12180kB slab_unreclaimable:10824kB kernel_stack:752kB pagetables:12524kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10139 all_unreclaimable? yes
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272252] lowmem_reserve[]: 0 0 0 0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272254] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272261] Node 0 DMA32: 1467*4kB 8*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8012kB
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272268] 1470 total pagecache pages
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272269] 0 pages in swap cache
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272270] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272271] Free swap = 0kB
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.272272] Total swap = 0kB
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277415] 985072 pages RAM
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277417] 24998 pages reserved
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277418] 3796 pages shared
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277419] 953299 pages non-shared
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277421] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277430] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277433] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277436] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277439] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277443] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277446] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277449] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277453] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277457] [ 790] 101 790 63477 599 0 0 0 rsyslogd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277460] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277463] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277466] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277469] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277472] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277475] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277478] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277481] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277484] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277488] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277491] [ 654] 1012 654 70287 5743 1 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277494] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277497] [ 658] 1012 658 135033 71439 0 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277500] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277503] [13468] 0 13468 89584 2504 0 0 0 php5-fpm
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277506] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277509] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277513] [ 8112] 1012 8112 68610 5660 0 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277516] [31572] 108 31572 6792 70 1 0 0 pickup
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277519] [ 9497] 1012 9497 890579 827488 1 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277522] [ 9595] 108 9595 6818 73 0 0 0 cleanup
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277526] [ 9596] 108 9596 6795 70 0 0 0 trivial-rewrite
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277529] [ 9597] 108 9597 11107 158 1 0 0 smtp
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277532] [10121] 1012 10121 68580 5604 0 0 0 php
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277535] Out of memory: Kill process 9497 (php) score 863 or sacrifice child
Jul 14 07:29:56 ip-10-61-174-46 kernel: [1313892.277545] Killed process 9497 (php) total-vm:3562316kB, anon-rss:3309928kB, file-rss:24kB
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918815] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918820] php cpuset=/ mems_allowed=0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918823] Pid: 11880, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918825] Call Trace:
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918833] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918836] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918839] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918843] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918848] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918852] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918857] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918860] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918864] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918868] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918872] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918875] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918879] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918886] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918890] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918893] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918897] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918900] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918905] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918909] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918913] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918916] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918918] Mem-Info:
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918920] Node 0 DMA per-cpu:
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918923] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918924] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918925] Node 0 DMA32 per-cpu:
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918928] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918929] CPU 1: hi: 186, btch: 31 usd: 11
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918933] active_anon:932645 inactive_anon:1393 isolated_anon:0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918934] active_file:0 inactive_file:8 isolated_file:0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918935] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918936] free:4093 slab_reclaimable:2594 slab_unreclaimable:2695
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918937] mapped:1358 shmem:1405 pagetables:3187 bounce:0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918939] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918947] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918951] Node 0 DMA32 free:8036kB min:7848kB low:9808kB high:11772kB active_anon:3730580kB inactive_anon:5572kB active_file:0kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:10376kB slab_unreclaimable:10780kB kernel_stack:768kB pagetables:12748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918960] lowmem_reserve[]: 0 0 0 0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918963] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918973] Node 0 DMA32: 787*4kB 105*8kB 109*16kB 8*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8036kB
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918982] 1445 total pagecache pages
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918984] 0 pages in swap cache
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918985] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918987] Free swap = 0kB
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.918988] Total swap = 0kB
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926194] 985072 pages RAM
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926196] 24998 pages reserved
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926198] 3787 pages shared
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926199] 953332 pages non-shared
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926201] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926209] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926212] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926215] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926218] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926221] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926224] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926226] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926229] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926233] [ 790] 101 790 63477 606 0 0 0 rsyslogd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926236] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926239] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926242] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926245] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926248] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926251] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926254] [ 1051] 0 1051 201627 844 0 0 0 collectd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926257] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926260] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926264] [15657] 0 15657 4777 59 0 0 0 cron
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926266] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926269] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926272] [ 658] 1012 658 135033 71439 1 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926275] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926278] [13468] 0 13468 89584 2511 0 0 0 php5-fpm
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926280] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926283] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926286] [ 8112] 1012 8112 68610 5659 1 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926289] [31572] 108 31572 6792 76 0 0 0 pickup
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926293] [10396] 1012 10396 888569 825488 1 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926296] [11776] 108 11776 6818 73 0 0 0 cleanup
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926299] [11777] 108 11777 6795 69 1 0 0 trivial-rewrite
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926302] [11778] 108 11778 11107 159 0 0 0 smtp
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926305] [11782] 108 11782 11107 158 0 0 0 smtp
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926308] [11786] 108 11786 11107 159 0 0 0 smtp
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926311] [11880] 1012 11880 70571 7579 0 0 0 php
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926314] Out of memory: Kill process 10396 (php) score 861 or sacrifice child
Jul 14 07:33:23 ip-10-61-174-46 kernel: [1314098.926327] Killed process 10396 (php) total-vm:3554276kB, anon-rss:3301952kB, file-rss:0kB
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116023] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116028] ntpd cpuset=/ mems_allowed=0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116031] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116033] Call Trace:
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116041] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116045] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116048] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116066] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116071] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116076] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116079] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116083] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116086] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116090] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116093] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116096] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116101] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116105] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116109] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116111] Mem-Info:
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116113] Node 0 DMA per-cpu:
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116116] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116118] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116119] Node 0 DMA32 per-cpu:
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116122] CPU 0: hi: 186, btch: 31 usd: 89
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116124] CPU 1: hi: 186, btch: 31 usd: 10
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116128] active_anon:932500 inactive_anon:1393 isolated_anon:0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116129] active_file:23 inactive_file:35 isolated_file:12
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116130] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116131] free:4043 slab_reclaimable:2625 slab_unreclaimable:2645
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116132] mapped:1390 shmem:1405 pagetables:3279 bounce:0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116134] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116142] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116146] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3730000kB inactive_anon:5572kB active_file:92kB inactive_file:140kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5560kB shmem:5620kB slab_reclaimable:10500kB slab_unreclaimable:10580kB kernel_stack:768kB pagetables:13116kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:194 all_unreclaimable? yes
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116155] lowmem_reserve[]: 0 0 0 0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116158] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116169] Node 0 DMA32: 1103*4kB 163*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116179] 1490 total pagecache pages
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116180] 0 pages in swap cache
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116182] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116184] Free swap = 0kB
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.116185] Total swap = 0kB
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121619] 985072 pages RAM
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121621] 24998 pages reserved
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121622] 3802 pages shared
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121623] 953219 pages non-shared
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121624] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121632] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121635] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121637] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121639] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121641] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121644] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121646] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121649] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121652] [ 790] 101 790 63477 606 1 0 0 rsyslogd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121655] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121657] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121659] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121662] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121664] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121666] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121668] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121671] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121673] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121675] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121677] [ 654] 1012 654 70351 5745 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121679] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121682] [ 658] 1012 658 135033 71439 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121684] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121686] [13468] 0 13468 89584 2529 1 0 0 php5-fpm
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121688] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121690] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121693] [ 8112] 1012 8112 68610 5662 1 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121695] [12047] 1012 12047 880712 817658 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121697] [ 970] 108 970 6792 69 0 0 0 pickup
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121700] [ 3659] 1012 3659 68580 5610 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121702] [ 3841] 1012 3841 1099 25 0 0 0 sh
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121704] [ 3842] 1012 3842 68825 5872 0 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121706] [ 3848] 1012 3848 1099 25 0 0 0 sh
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121708] [ 3849] 1012 3849 4776 57 0 0 0 git
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121710] [ 3850] 1012 3850 67197 4289 1 0 0 php
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121712] Out of memory: Kill process 12047 (php) score 853 or sacrifice child
Jul 14 08:29:38 ip-10-61-174-46 kernel: [1317474.121722] Killed process 12047 (php) total-vm:3522848kB, anon-rss:3270632kB, file-rss:0kB
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660613] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660617] php cpuset=/ mems_allowed=0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660620] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660621] Call Trace:
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660629] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660631] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660633] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660636] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660639] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660643] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660645] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660648] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660652] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660654] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660657] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660660] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660662] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660665] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660668] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660670] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660673] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660676] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660679] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660682] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660684] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660686] Mem-Info:
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660687] Node 0 DMA per-cpu:
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660689] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660690] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660691] Node 0 DMA32 per-cpu:
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660693] CPU 0: hi: 186, btch: 31 usd: 4
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660694] CPU 1: hi: 186, btch: 31 usd: 57
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660697] active_anon:932670 inactive_anon:1393 isolated_anon:0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660698] active_file:10 inactive_file:16 isolated_file:0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660698] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660699] free:4034 slab_reclaimable:2701 slab_unreclaimable:2666
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660700] mapped:1364 shmem:1405 pagetables:3170 bounce:0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660701] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660708] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660711] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3730680kB inactive_anon:5572kB active_file:40kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10804kB slab_unreclaimable:10664kB kernel_stack:744kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:188 all_unreclaimable? yes
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660717] lowmem_reserve[]: 0 0 0 0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660720] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660727] Node 0 DMA32: 1438*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660733] 1470 total pagecache pages
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660734] 0 pages in swap cache
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660735] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660737] Free swap = 0kB
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.660737] Total swap = 0kB
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666155] 985072 pages RAM
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666158] 24998 pages reserved
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666159] 3800 pages shared
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666160] 953288 pages non-shared
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666162] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666171] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666175] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666178] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666181] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666184] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666188] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666191] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666194] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666198] [ 790] 101 790 63477 606 0 0 0 rsyslogd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666201] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666204] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666208] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666211] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666214] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666217] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666220] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666223] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666226] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666229] [15657] 0 15657 4777 60 1 0 0 cron
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666233] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666236] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666239] [ 658] 1012 658 135033 71439 0 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666242] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666245] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666248] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666251] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666254] [ 8112] 1012 8112 68610 5663 0 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666257] [ 3905] 1012 3905 885361 822230 1 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666260] [29427] 108 29427 6792 69 0 0 0 pickup
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666263] [14893] 1012 14893 68580 5604 0 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666266] [15016] 1012 15016 1099 25 0 0 0 sh
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666270] [15017] 1012 15017 68825 5869 1 0 0 php
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666272] Out of memory: Kill process 3905 (php) score 858 or sacrifice child
Jul 14 10:21:45 ip-10-61-174-46 kernel: [1324200.666284] Killed process 3905 (php) total-vm:3541444kB, anon-rss:3288920kB, file-rss:0kB
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912139] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912144] php cpuset=/ mems_allowed=0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912147] Pid: 15078, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912149] Call Trace:
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912169] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912172] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912174] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912177] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912180] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912182] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912185] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912187] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912191] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912193] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912195] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912198] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912200] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912202] Mem-Info:
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912203] Node 0 DMA per-cpu:
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912205] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912207] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912208] Node 0 DMA32 per-cpu:
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912209] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912211] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912213] active_anon:933170 inactive_anon:1393 isolated_anon:0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912214] active_file:13 inactive_file:15 isolated_file:0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912215] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912215] free:4048 slab_reclaimable:2435 slab_unreclaimable:2629
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912216] mapped:1358 shmem:1405 pagetables:3076 bounce:0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912217] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912224] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912227] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732680kB inactive_anon:5572kB active_file:52kB inactive_file:60kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9740kB slab_unreclaimable:10516kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14791 all_unreclaimable? yes
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912234] lowmem_reserve[]: 0 0 0 0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912236] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912243] Node 0 DMA32: 799*4kB 166*8kB 72*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912249] 1463 total pagecache pages
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912251] 0 pages in swap cache
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912252] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912253] Free swap = 0kB
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.912254] Total swap = 0kB
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917547] 985072 pages RAM
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917549] 24998 pages reserved
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917550] 3793 pages shared
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917551] 953355 pages non-shared
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917553] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917563] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917566] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917570] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917573] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917576] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917579] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917582] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917586] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917590] [ 790] 101 790 63477 606 1 0 0 rsyslogd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917593] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917596] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917599] [ 975] 0 975 6276 79 0 0 0 master
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917602] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917605] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917609] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917612] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917615] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917618] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917621] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917624] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917628] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917631] [ 658] 1012 658 135033 71438 1 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917634] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917637] [13468] 0 13468 89584 2536 0 0 0 php5-fpm
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917640] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917643] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917646] [ 8112] 1012 8112 68610 5663 1 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917649] [29427] 108 29427 6792 69 0 0 0 pickup
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917652] [15078] 1012 15078 891627 828609 0 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917656] [17814] 1012 17814 68580 5604 1 0 0 php
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917658] Out of memory: Kill process 15078 (php) score 864 or sacrifice child
Jul 14 10:29:40 ip-10-61-174-46 kernel: [1324675.917669] Killed process 15078 (php) total-vm:3566508kB, anon-rss:3314348kB, file-rss:88kB
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916137] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916142] php cpuset=/ mems_allowed=0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916145] Pid: 17913, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916148] Call Trace:
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916155] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916158] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916161] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916169] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916174] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916185] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916192] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916195] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916198] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916201] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916204] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916211] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916214] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916218] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916221] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916224] Mem-Info:
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916225] Node 0 DMA per-cpu:
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916228] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916230] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916231] Node 0 DMA32 per-cpu:
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916235] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916239] active_anon:932922 inactive_anon:1393 isolated_anon:0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916240] active_file:0 inactive_file:19 isolated_file:0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916241] unevictable:0 dirty:9 writeback:0 unstable:0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916242] free:4045 slab_reclaimable:2697 slab_unreclaimable:2625
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916242] mapped:1364 shmem:1405 pagetables:3075 bounce:0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916244] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916252] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916255] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3731688kB inactive_anon:5572kB active_file:0kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:36kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10788kB slab_unreclaimable:10500kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:47623 all_unreclaimable? yes
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916262] lowmem_reserve[]: 0 0 0 0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916264] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916271] Node 0 DMA32: 952*4kB 234*8kB 15*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7968kB
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916278] 1436 total pagecache pages
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916279] 0 pages in swap cache
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916280] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916281] Free swap = 0kB
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.916282] Total swap = 0kB
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921609] 985072 pages RAM
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921611] 24998 pages reserved
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921613] 3781 pages shared
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921614] 953334 pages non-shared
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921616] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921624] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921628] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921631] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921635] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921638] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921641] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921644] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921647] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921652] [ 790] 101 790 63477 606 0 0 0 rsyslogd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921655] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921658] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921661] [ 975] 0 975 6276 79 1 0 0 master
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921664] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921667] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921670] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921674] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921677] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921680] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921683] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921686] [ 654] 1012 654 70351 5746 0 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921689] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921692] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921695] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921698] [13468] 0 13468 89584 2521 0 0 0 php5-fpm
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921701] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921704] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921707] [ 8112] 1012 8112 68610 5663 1 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921710] [17913] 1012 17913 892089 828972 1 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921714] [30131] 108 30131 6792 69 1 0 0 pickup
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921717] [ 7975] 1012 7975 68580 5605 0 0 0 php
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921720] Out of memory: Kill process 17913 (php) score 865 or sacrifice child
Jul 14 11:33:26 ip-10-61-174-46 kernel: [1328501.921730] Killed process 17913 (php) total-vm:3568356kB, anon-rss:3315888kB, file-rss:0kB
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600908] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600912] php cpuset=/ mems_allowed=0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600915] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600916] Call Trace:
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600924] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600926] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600928] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600931] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600934] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600938] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600940] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600943] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600948] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600950] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600953] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600956] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600958] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600961] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600963] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600966] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600969] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600973] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600976] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600979] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600981] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600983] Mem-Info:
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600984] Node 0 DMA per-cpu:
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600986] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600987] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600988] Node 0 DMA32 per-cpu:
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600990] CPU 0: hi: 186, btch: 31 usd: 30
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600991] CPU 1: hi: 186, btch: 31 usd: 27
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600994] active_anon:932953 inactive_anon:1393 isolated_anon:0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600995] active_file:6 inactive_file:10 isolated_file:0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600995] unevictable:0 dirty:22 writeback:0 unstable:0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600996] free:4035 slab_reclaimable:2618 slab_unreclaimable:2645
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600997] mapped:1357 shmem:1405 pagetables:3048 bounce:0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.600998] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601005] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601008] Node 0 DMA32 free:7804kB min:7848kB low:9808kB high:11772kB active_anon:3731812kB inactive_anon:5572kB active_file:24kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10472kB slab_unreclaimable:10580kB kernel_stack:720kB pagetables:12192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:153 all_unreclaimable? yes
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601014] lowmem_reserve[]: 0 0 0 0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601017] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601023] Node 0 DMA32: 871*4kB 233*8kB 25*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601030] 1401 total pagecache pages
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601031] 0 pages in swap cache
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601033] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601034] Free swap = 0kB
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.601035] Total swap = 0kB
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606230] 985072 pages RAM
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606233] 24998 pages reserved
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606234] 3788 pages shared
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606235] 953292 pages non-shared
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606237] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606246] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606250] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606253] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606256] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606259] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606262] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606266] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606269] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606273] [ 790] 101 790 63477 606 0 0 0 rsyslogd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606276] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606279] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606282] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606285] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606288] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606291] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606294] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606297] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606301] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606304] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606307] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606310] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606313] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606316] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606319] [13468] 0 13468 89584 2506 1 0 0 php5-fpm
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606322] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606325] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606328] [ 8112] 1012 8112 68610 5663 0 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606331] [ 8100] 1012 8100 892207 829060 1 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606335] [24435] 108 24435 6792 69 0 0 0 pickup
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606338] [26729] 1012 26729 68580 5605 1 0 0 php
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606340] Out of memory: Kill process 8100 (php) score 865 or sacrifice child
Jul 14 14:21:46 ip-10-61-174-46 kernel: [1338601.606351] Killed process 8100 (php) total-vm:3568828kB, anon-rss:3316240kB, file-rss:0kB
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532149] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532153] php cpuset=/ mems_allowed=0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532156] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532158] Call Trace:
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532165] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532167] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532169] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532172] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532177] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532180] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532183] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532185] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532188] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532192] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532197] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532199] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532202] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532205] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532207] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532210] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532214] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532216] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532219] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532222] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532223] Mem-Info:
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532224] Node 0 DMA per-cpu:
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532226] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532228] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532229] Node 0 DMA32 per-cpu:
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532230] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532232] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532234] active_anon:932969 inactive_anon:1393 isolated_anon:0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532235] active_file:0 inactive_file:0 isolated_file:0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532235] unevictable:0 dirty:22 writeback:0 unstable:0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532236] free:4048 slab_reclaimable:2558 slab_unreclaimable:2612
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532237] mapped:1377 shmem:1405 pagetables:3070 bounce:0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532238] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532245] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532248] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731876kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:10232kB slab_unreclaimable:10448kB kernel_stack:728kB pagetables:12280kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532254] lowmem_reserve[]: 0 0 0 0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532257] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532264] Node 0 DMA32: 754*4kB 171*8kB 73*16kB 8*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532270] 1467 total pagecache pages
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532272] 0 pages in swap cache
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532273] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532274] Free swap = 0kB
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.532275] Total swap = 0kB
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537452] 985072 pages RAM
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537455] 24998 pages reserved
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537456] 3799 pages shared
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537457] 953339 pages non-shared
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537459] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537468] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537471] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537474] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537478] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537481] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537484] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537487] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537491] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537495] [ 790] 101 790 63477 606 1 0 0 rsyslogd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537498] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537501] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537505] [ 975] 0 975 6276 79 1 0 0 master
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537508] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537511] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537514] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537517] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537520] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537523] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537526] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537530] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537533] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537536] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537539] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537542] [13468] 0 13468 89584 2521 0 0 0 php5-fpm
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537545] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537548] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537551] [ 8112] 1012 8112 68610 5664 1 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537554] [24435] 108 24435 6792 75 0 0 0 pickup
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537558] [26872] 1012 26872 892202 829071 0 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537561] [28787] 1012 28787 68580 5604 0 0 0 php
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537563] Out of memory: Kill process 26872 (php) score 865 or sacrifice child
Jul 14 14:29:39 ip-10-61-174-46 kernel: [1339075.537574] Killed process 26872 (php) total-vm:3568808kB, anon-rss:3316284kB, file-rss:0kB
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032154] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032158] php cpuset=/ mems_allowed=0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032161] Pid: 28908, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032163] Call Trace:
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032169] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032172] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032174] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032177] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032180] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032183] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032185] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032188] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032192] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032197] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032202] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032205] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032207] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032210] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032212] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032214] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032217] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032220] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032222] Mem-Info:
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032223] Node 0 DMA per-cpu:
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032225] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032226] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032227] Node 0 DMA32 per-cpu:
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032229] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032230] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032233] active_anon:933180 inactive_anon:1393 isolated_anon:0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032233] active_file:0 inactive_file:0 isolated_file:0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032234] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032235] free:4048 slab_reclaimable:2456 slab_unreclaimable:2582
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032235] mapped:1351 shmem:1405 pagetables:3075 bounce:0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032237] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032243] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032246] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732720kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5404kB shmem:5620kB slab_reclaimable:9824kB slab_unreclaimable:10328kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032253] lowmem_reserve[]: 0 0 0 0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032255] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032262] Node 0 DMA32: 870*4kB 211*8kB 15*16kB 10*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032269] 1461 total pagecache pages
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032270] 0 pages in swap cache
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032271] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032272] Free swap = 0kB
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.032273] Total swap = 0kB
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037497] 985072 pages RAM
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037500] 24998 pages reserved
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037501] 3784 pages shared
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037502] 953341 pages non-shared
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037504] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037513] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037517] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037520] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037523] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037526] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037530] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037533] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037536] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037540] [ 790] 101 790 63477 610 1 0 0 rsyslogd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037544] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037547] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037550] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037553] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037556] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037560] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037563] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037566] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037569] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037572] [15657] 0 15657 4777 60 1 0 0 cron
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037575] [ 654] 1012 654 70351 5745 0 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037579] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037582] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037585] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037588] [13468] 0 13468 89584 2506 1 0 0 php5-fpm
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037591] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037594] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037597] [ 8112] 1012 8112 68610 5685 1 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037600] [28908] 1012 28908 892404 829245 1 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037603] [ 4158] 108 4158 6792 69 0 0 0 pickup
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037607] [11385] 1012 11385 68580 5605 1 0 0 php
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037610] Out of memory: Kill process 28908 (php) score 865 or sacrifice child
Jul 14 15:29:57 ip-10-61-174-46 kernel: [1342693.037620] Killed process 28908 (php) total-vm:3569616kB, anon-rss:3316980kB, file-rss:0kB
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156493] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156498] php cpuset=/ mems_allowed=0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156502] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156504] Call Trace:
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156512] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156515] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156518] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156522] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156527] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156531] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156534] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156537] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156540] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156543] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156546] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156548] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156551] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156554] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156556] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156560] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156563] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156566] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156568] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156571] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156573] Mem-Info:
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156574] Node 0 DMA per-cpu:
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156576] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156577] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156578] Node 0 DMA32 per-cpu:
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156580] CPU 0: hi: 186, btch: 31 usd: 46
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156582] CPU 1: hi: 186, btch: 31 usd: 30
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156585] active_anon:933102 inactive_anon:1393 isolated_anon:0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156586] active_file:8 inactive_file:0 isolated_file:0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156587] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156588] free:4026 slab_reclaimable:2440 slab_unreclaimable:2627
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156589] mapped:1376 shmem:1405 pagetables:3057 bounce:0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156591] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156598] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156601] Node 0 DMA32 free:7768kB min:7848kB low:9808kB high:11772kB active_anon:3732408kB inactive_anon:5572kB active_file:32kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5504kB shmem:5620kB slab_reclaimable:9760kB slab_unreclaimable:10508kB kernel_stack:728kB pagetables:12228kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:173 all_unreclaimable? yes
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156607] lowmem_reserve[]: 0 0 0 0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156610] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156617] Node 0 DMA32: 916*4kB 219*8kB 15*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7768kB
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156623] 1412 total pagecache pages
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156625] 0 pages in swap cache
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156626] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156627] Free swap = 0kB
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.156628] Total swap = 0kB
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161964] 985072 pages RAM
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161966] 24998 pages reserved
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161968] 3776 pages shared
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161969] 953284 pages non-shared
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161971] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161979] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161983] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161986] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161989] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161992] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161996] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.161999] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162002] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162006] [ 790] 101 790 63477 610 1 0 0 rsyslogd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162009] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162012] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162015] [ 975] 0 975 6276 79 0 0 0 master
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162018] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162022] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162025] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162028] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162031] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162034] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162037] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162040] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162043] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162047] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162050] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162053] [13468] 0 13468 89584 2538 0 0 0 php5-fpm
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162056] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162059] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162062] [ 8112] 1012 8112 68610 5684 0 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162065] [ 4158] 108 4158 6792 69 0 0 0 pickup
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162069] [11505] 1012 11505 892239 829168 1 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162072] [12244] 1012 12244 68580 5604 0 0 0 php
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162074] Out of memory: Kill process 11505 (php) score 865 or sacrifice child
Jul 14 15:33:25 ip-10-61-174-46 kernel: [1342901.162086] Killed process 11505 (php) total-vm:3568956kB, anon-rss:3316672kB, file-rss:0kB
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316428] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316432] php cpuset=/ mems_allowed=0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316435] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316436] Call Trace:
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316443] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316445] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316448] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316450] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316454] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316457] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316459] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316462] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316466] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316469] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316471] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316475] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316477] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316480] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316482] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316485] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316487] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316490] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316494] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316496] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316499] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316500] Mem-Info:
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316502] Node 0 DMA per-cpu:
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316504] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316505] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316506] Node 0 DMA32 per-cpu:
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316508] CPU 0: hi: 186, btch: 31 usd: 30
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316509] CPU 1: hi: 186, btch: 31 usd: 76
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316512] active_anon:932683 inactive_anon:1393 isolated_anon:0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316512] active_file:0 inactive_file:0 isolated_file:0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316513] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316514] free:4036 slab_reclaimable:2826 slab_unreclaimable:2610
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316514] mapped:1357 shmem:1405 pagetables:3073 bounce:0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316516] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316522] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316525] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3730732kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:11304kB slab_unreclaimable:10440kB kernel_stack:728kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:128 all_unreclaimable? yes
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316532] lowmem_reserve[]: 0 0 0 0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316534] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316541] Node 0 DMA32: 1381*4kB 21*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7740kB
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316548] 1436 total pagecache pages
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316549] 0 pages in swap cache
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316550] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316551] Free swap = 0kB
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.316552] Total swap = 0kB
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321859] 985072 pages RAM
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321862] 24998 pages reserved
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321863] 3787 pages shared
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321864] 953273 pages non-shared
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321866] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321875] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321878] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321882] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321885] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321888] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321891] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321895] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321898] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321902] [ 790] 101 790 63477 610 0 0 0 rsyslogd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321906] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321909] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321912] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321915] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321918] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321921] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321925] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321928] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321931] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321934] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321937] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321940] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321943] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321946] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321950] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321953] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321955] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321959] [ 8112] 1012 8112 68610 5685 1 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321962] [12340] 1012 12340 891860 828744 0 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321965] [20087] 108 20087 6792 69 0 0 0 pickup
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321969] [25981] 1012 25981 68580 5604 1 0 0 php
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321971] Out of memory: Kill process 12340 (php) score 864 or sacrifice child
Jul 14 16:21:47 ip-10-61-174-46 kernel: [1345803.321982] Killed process 12340 (php) total-vm:3567440kB, anon-rss:3314976kB, file-rss:0kB
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863981] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863986] php cpuset=/ mems_allowed=0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863989] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863991] Call Trace:
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.863999] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864002] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864005] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864009] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864015] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864019] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864023] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864026] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864030] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864033] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864037] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864040] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864043] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864047] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864052] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864067] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864070] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864072] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864077] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864079] Mem-Info:
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864081] Node 0 DMA per-cpu:
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864083] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864085] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864086] Node 0 DMA32 per-cpu:
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864089] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864090] CPU 1: hi: 186, btch: 31 usd: 30
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864094] active_anon:933012 inactive_anon:1393 isolated_anon:0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864095] active_file:0 inactive_file:22 isolated_file:0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864096] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864097] free:4043 slab_reclaimable:2580 slab_unreclaimable:2606
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864098] mapped:1352 shmem:1405 pagetables:3072 bounce:0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864100] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864108] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864112] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3732048kB inactive_anon:5572kB active_file:0kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:10320kB slab_unreclaimable:10424kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10 all_unreclaimable? yes
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864121] lowmem_reserve[]: 0 0 0 0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864124] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864134] Node 0 DMA32: 810*4kB 160*8kB 32*16kB 22*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7848kB
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864144] 1427 total pagecache pages
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864145] 0 pages in swap cache
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864147] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864157] Free swap = 0kB
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.864158] Total swap = 0kB
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869957] 985072 pages RAM
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869960] 24998 pages reserved
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869961] 3787 pages shared
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869962] 953324 pages non-shared
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869964] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869972] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869976] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869979] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869982] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869985] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869988] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869991] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869994] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.869998] [ 790] 101 790 63477 610 0 0 0 rsyslogd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870001] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870003] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870006] [ 975] 0 975 6276 79 0 0 0 master
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870009] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870012] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870015] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870018] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870021] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870025] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870028] [15657] 0 15657 4777 59 0 0 0 cron
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870031] [ 654] 1012 654 70351 5745 0 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870034] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870037] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870040] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870044] [13468] 0 13468 89584 2506 0 0 0 php5-fpm
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870047] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870049] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870053] [ 8112] 1012 8112 68610 5684 0 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870056] [20087] 108 20087 6792 69 1 0 0 pickup
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870059] [26079] 1012 26079 892139 829074 1 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870061] [28333] 1012 28333 68580 5603 1 0 0 php
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870064] Out of memory: Kill process 26079 (php) score 865 or sacrifice child
Jul 14 16:29:40 ip-10-61-174-46 kernel: [1346275.870075] Killed process 26079 (php) total-vm:3568556kB, anon-rss:3316296kB, file-rss:0kB
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544136] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544140] php cpuset=/ mems_allowed=0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544144] Pid: 29039, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544146] Call Trace:
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544157] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544160] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544168] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544172] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544175] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544183] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544185] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544191] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544193] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544196] [<ffffffff811425e0>] ? find_vma+0x30/0x70
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544201] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544203] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544206] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544209] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544210] Mem-Info:
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544211] Node 0 DMA per-cpu:
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544216] Node 0 DMA32 per-cpu:
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544221] active_anon:932418 inactive_anon:1393 isolated_anon:20
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544222] active_file:13 inactive_file:14 isolated_file:0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544223] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544223] free:4048 slab_reclaimable:2817 slab_unreclaimable:2708
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544224] mapped:1340 shmem:1405 pagetables:3227 bounce:0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544233] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729672kB inactive_anon:5572kB active_file:52kB inactive_file:56kB unevictable:0kB isolated(anon):80kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5360kB shmem:5620kB slab_reclaimable:11268kB slab_unreclaimable:10832kB kernel_stack:784kB pagetables:12908kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10988 all_unreclaimable? yes
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544242] lowmem_reserve[]: 0 0 0 0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544251] Node 0 DMA32: 1480*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544258] 1448 total pagecache pages
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544259] 0 pages in swap cache
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544260] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544261] Free swap = 0kB
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.544262] Total swap = 0kB
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549504] 985072 pages RAM
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549506] 24998 pages reserved
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549508] 3791 pages shared
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549509] 953301 pages non-shared
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549511] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549519] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549522] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549526] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549529] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549532] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549535] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549539] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549542] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549546] [ 790] 101 790 63477 610 1 0 0 rsyslogd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549549] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549552] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549555] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549558] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549561] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549564] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549567] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549570] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549574] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549577] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549580] [ 654] 1012 654 70351 5745 1 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549583] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549586] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549589] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549592] [13468] 0 13468 89584 2517 0 0 0 php5-fpm
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549595] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549598] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549601] [ 8112] 1012 8112 68610 5685 1 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549605] [29039] 1012 29039 885137 822059 1 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549608] [ 2228] 108 2228 6792 69 1 0 0 pickup
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549611] [ 4421] 108 4421 6818 74 1 0 0 cleanup
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549614] [ 4422] 108 4422 6795 71 0 0 0 trivial-rewrite
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549617] [ 4423] 108 4423 11107 158 1 0 0 smtp
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549620] [ 4757] 1012 4757 68580 5606 1 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549623] [ 4849] 1012 4849 1099 24 0 0 0 sh
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549626] [ 4850] 1012 4850 68982 6013 0 0 0 php
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549629] [ 4873] 1012 4873 1099 25 1 0 0 sh
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549632] [ 4876] 1012 4876 4798 69 0 0 0 git
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549635] Out of memory: Kill process 29039 (php) score 858 or sacrifice child
Jul 14 18:48:52 ip-10-61-174-46 kernel: [1354627.549646] Killed process 29039 (php) total-vm:3540548kB, anon-rss:3288236kB, file-rss:0kB
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003469] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003473] php cpuset=/ mems_allowed=0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003476] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003477] Call Trace:
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003485] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003487] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003489] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003492] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003495] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003499] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003501] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003504] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003507] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003510] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003512] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003514] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003517] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003521] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003524] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003528] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003531] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003534] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003537] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003539] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003542] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003543] Mem-Info:
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003545] Node 0 DMA per-cpu:
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003547] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003548] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003549] Node 0 DMA32 per-cpu:
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003551] CPU 0: hi: 186, btch: 31 usd: 61
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003552] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003555] active_anon:932982 inactive_anon:1393 isolated_anon:0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003556] active_file:40 inactive_file:0 isolated_file:0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003557] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003557] free:4022 slab_reclaimable:2444 slab_unreclaimable:2626
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003558] mapped:1356 shmem:1405 pagetables:3170 bounce:0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003560] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003566] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003569] Node 0 DMA32 free:7752kB min:7848kB low:9808kB high:11772kB active_anon:3731928kB inactive_anon:5572kB active_file:160kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9776kB slab_unreclaimable:10504kB kernel_stack:760kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:154 all_unreclaimable? yes
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003576] lowmem_reserve[]: 0 0 0 0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003578] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003585] Node 0 DMA32: 1426*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7752kB
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003591] 1435 total pagecache pages
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003593] 0 pages in swap cache
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003594] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003595] Free swap = 0kB
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.003596] Total swap = 0kB
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008774] 985072 pages RAM
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008777] 24998 pages reserved
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008778] 3792 pages shared
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008779] 953306 pages non-shared
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008781] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008790] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008793] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008797] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008800] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008803] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008806] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008810] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008813] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008817] [ 790] 101 790 63477 610 0 0 0 rsyslogd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008820] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008823] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008827] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008830] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008833] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008836] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008839] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008842] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008845] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008849] [15657] 0 15657 4777 59 0 0 0 cron
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008852] [ 654] 1012 654 70351 5746 0 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008855] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008858] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008861] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008865] [13468] 0 13468 89584 2520 1 0 0 php5-fpm
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008868] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008871] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008874] [ 5841] 1012 5841 68618 5641 1 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008877] [11561] 108 11561 6792 70 1 0 0 pickup
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008880] [12406] 1012 12406 882237 819092 1 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008883] [19463] 108 19463 6795 71 1 0 0 trivial-rewrite
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008886] [19954] 1012 19954 78096 15119 1 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008889] [20129] 108 20129 6818 74 1 0 0 cleanup
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008893] [20130] 108 20130 11107 159 0 0 0 smtp
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008896] [20131] 1012 20131 12484 151 0 0 0 php
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008898] Out of memory: Kill process 12406 (php) score 854 or sacrifice child
Jul 14 19:30:02 ip-10-61-174-46 kernel: [1357098.008910] Killed process 12406 (php) total-vm:3528948kB, anon-rss:3276368kB, file-rss:0kB
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284126] php cpuset=/ mems_allowed=0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284129] Pid: 20163, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284131] Call Trace:
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284142] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284145] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284148] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284154] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284157] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284160] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284163] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284166] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284168] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284170] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284173] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284175] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284178] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284181] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284183] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284185] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284188] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284191] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284192] Mem-Info:
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284194] Node 0 DMA per-cpu:
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284196] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284197] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284198] Node 0 DMA32 per-cpu:
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284200] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284201] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284204] active_anon:932816 inactive_anon:1393 isolated_anon:0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284204] active_file:21 inactive_file:22 isolated_file:0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284205] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284206] free:4048 slab_reclaimable:2543 slab_unreclaimable:2654
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284206] mapped:1392 shmem:1405 pagetables:3213 bounce:0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284208] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284214] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284217] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731264kB inactive_anon:5572kB active_file:84kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5568kB shmem:5620kB slab_reclaimable:10172kB slab_unreclaimable:10616kB kernel_stack:776kB pagetables:12852kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2079 all_unreclaimable? yes
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284224] lowmem_reserve[]: 0 0 0 0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284226] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284233] Node 0 DMA32: 973*4kB 213*8kB 15*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284240] 1469 total pagecache pages
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284241] 0 pages in swap cache
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284242] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284243] Free swap = 0kB
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.284244] Total swap = 0kB
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289638] 985072 pages RAM
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289640] 24998 pages reserved
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289641] 3802 pages shared
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289642] 953322 pages non-shared
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289644] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289653] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289657] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289660] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289663] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289666] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289670] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289673] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289676] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289680] [ 790] 101 790 63477 610 0 0 0 rsyslogd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289683] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289687] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289690] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289693] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289696] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289699] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289702] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289705] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289709] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289712] [15657] 0 15657 4777 59 0 0 0 cron
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289715] [ 654] 1012 654 70351 5746 1 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289718] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289721] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289724] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289727] [13468] 0 13468 89584 2517 0 0 0 php5-fpm
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289731] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289733] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289737] [ 5841] 1012 5841 68618 5645 1 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289740] [11561] 108 11561 6792 76 1 0 0 pickup
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289743] [20163] 1012 20163 891143 828169 0 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289746] [20731] 108 20731 6818 74 1 0 0 cleanup
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289749] [20732] 108 20732 6795 71 0 0 0 trivial-rewrite
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289752] [20733] 108 20733 11107 158 1 0 0 smtp
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289755] [21016] 108 21016 11107 158 1 0 0 smtp
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289758] [21020] 108 21020 11107 159 0 0 0 smtp
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289761] [21024] 108 21024 11107 157 1 0 0 smtp
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289764] [21148] 1012 21148 68580 5603 1 0 0 php
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289767] Out of memory: Kill process 20163 (php) score 864 or sacrifice child
Jul 14 19:33:28 ip-10-61-174-46 kernel: [1357304.289778] Killed process 20163 (php) total-vm:3564572kB, anon-rss:3312588kB, file-rss:88kB
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506553] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506556] php cpuset=/ mems_allowed=0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506559] Pid: 21225, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506561] Call Trace:
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506568] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506570] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506573] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506575] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506579] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506582] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506584] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506587] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506590] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506593] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506596] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506598] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506601] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506603] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506606] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506609] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506611] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506615] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506618] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506619] Mem-Info:
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506620] Node 0 DMA per-cpu:
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506622] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506623] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506625] Node 0 DMA32 per-cpu:
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506626] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506628] CPU 1: hi: 186, btch: 31 usd: 14
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506630] active_anon:932900 inactive_anon:1393 isolated_anon:0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506631] active_file:0 inactive_file:2 isolated_file:0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506632] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506632] free:4048 slab_reclaimable:2722 slab_unreclaimable:2631
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506633] mapped:1350 shmem:1405 pagetables:3054 bounce:0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506634] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506641] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506644] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731600kB inactive_anon:5572kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:10888kB slab_unreclaimable:10524kB kernel_stack:728kB pagetables:12216kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:41 all_unreclaimable? yes
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506650] lowmem_reserve[]: 0 0 0 0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506653] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506660] Node 0 DMA32: 1420*4kB 6*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7888kB
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506666] 1419 total pagecache pages
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506667] 0 pages in swap cache
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506669] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506670] Free swap = 0kB
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.506671] Total swap = 0kB
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511815] 985072 pages RAM
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511817] 24998 pages reserved
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511819] 3788 pages shared
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511820] 953328 pages non-shared
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511822] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511831] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511835] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511838] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511841] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511844] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511847] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511851] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511854] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511858] [ 790] 101 790 63477 615 1 0 0 rsyslogd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511862] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511865] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511868] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511871] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511875] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511878] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511881] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511884] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511887] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511890] [15657] 0 15657 4777 60 1 0 0 cron
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511894] [ 654] 1012 654 70351 5747 1 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511897] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511900] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511903] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511906] [13468] 0 13468 89584 2502 1 0 0 php5-fpm
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511909] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511912] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511915] [ 5841] 1012 5841 68618 5647 1 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511918] [21225] 1012 21225 891926 828997 0 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511921] [ 757] 108 757 6792 70 1 0 0 pickup
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511925] [ 5308] 1012 5308 68580 5606 1 0 0 php
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511928] Out of memory: Kill process 21225 (php) score 865 or sacrifice child
Jul 14 20:21:47 ip-10-61-174-46 kernel: [1360203.511939] Killed process 21225 (php) total-vm:3567704kB, anon-rss:3315964kB, file-rss:24kB
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244161] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244166] ntpd cpuset=/ mems_allowed=0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244169] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244171] Call Trace:
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244180] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244183] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244186] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244190] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244196] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244199] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244204] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244206] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244210] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244213] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244217] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244220] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244223] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244228] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244232] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244237] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244239] Mem-Info:
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244240] Node 0 DMA per-cpu:
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244243] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244245] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244246] Node 0 DMA32 per-cpu:
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244248] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244250] CPU 1: hi: 186, btch: 31 usd: 27
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244254] active_anon:932972 inactive_anon:1393 isolated_anon:0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244255] active_file:0 inactive_file:21 isolated_file:0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244256] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244257] free:4048 slab_reclaimable:2549 slab_unreclaimable:2638
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244257] mapped:1361 shmem:1405 pagetables:3072 bounce:0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244260] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244269] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244273] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731888kB inactive_anon:5572kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10196kB slab_unreclaimable:10552kB kernel_stack:752kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:34 all_unreclaimable? yes
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244281] lowmem_reserve[]: 0 0 0 0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244285] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244293] Node 0 DMA32: 1426*4kB 11*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244301] 1456 total pagecache pages
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244302] 0 pages in swap cache
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244303] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244304] Free swap = 0kB
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.244305] Total swap = 0kB
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249906] 985072 pages RAM
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249909] 24998 pages reserved
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249910] 3786 pages shared
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249911] 953321 pages non-shared
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249913] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249922] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249925] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249929] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249932] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249935] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249938] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249942] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249945] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249949] [ 790] 101 790 63477 615 1 0 0 rsyslogd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249952] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249955] [ 831] 0 831 4226 41 1 0 0 atd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249959] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249962] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249965] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249968] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249971] [ 1051] 0 1051 201629 847 1 0 0 collectd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249974] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249977] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249980] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249984] [ 654] 1012 654 70351 5747 0 0 0 php
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249987] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249990] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249993] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249996] [13468] 0 13468 89584 2539 1 0 0 php5-fpm
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.249999] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250002] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250005] [ 5841] 1012 5841 68618 5647 1 0 0 php
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250008] [ 757] 108 757 6792 76 0 0 0 pickup
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250012] [12910] 1012 12910 897434 834204 1 0 0 php
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250015] [13494] 108 13494 6818 73 0 0 0 cleanup
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250018] [13495] 108 13495 6795 69 0 0 0 trivial-rewrite
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250021] [13496] 108 13496 11107 158 0 0 0 smtp
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250024] [13582] 108 13582 11107 159 1 0 0 smtp
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250027] Out of memory: Kill process 12910 (php) score 870 or sacrifice child
Jul 14 20:47:33 ip-10-61-174-46 kernel: [1361749.250039] Killed process 12910 (php) total-vm:3589736kB, anon-rss:3336816kB, file-rss:0kB
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825357] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825362] php5-fpm cpuset=/ mems_allowed=0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825364] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825366] Call Trace:
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825373] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825376] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825378] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825381] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825384] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825387] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825389] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825392] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825396] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825399] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825402] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825405] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825407] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825410] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825413] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825416] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825421] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825424] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825427] [<ffffffff8117ad6e>] ? fput+0xe/0x30
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825429] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825431] Mem-Info:
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825432] Node 0 DMA per-cpu:
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825434] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825436] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825437] Node 0 DMA32 per-cpu:
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825438] CPU 0: hi: 186, btch: 31 usd: 44
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825440] CPU 1: hi: 186, btch: 31 usd: 35
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825442] active_anon:932734 inactive_anon:1393 isolated_anon:0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825443] active_file:0 inactive_file:9 isolated_file:4
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825444] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825444] free:4032 slab_reclaimable:2349 slab_unreclaimable:2641
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825445] mapped:1351 shmem:1405 pagetables:3379 bounce:0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825446] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825453] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825456] Node 0 DMA32 free:7792kB min:7848kB low:9808kB high:11772kB active_anon:3730936kB inactive_anon:5572kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):16kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5404kB shmem:5620kB slab_reclaimable:9396kB slab_unreclaimable:10564kB kernel_stack:824kB pagetables:13516kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:177 all_unreclaimable? yes
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825463] lowmem_reserve[]: 0 0 0 0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825465] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825472] Node 0 DMA32: 724*4kB 288*8kB 29*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7712kB
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825479] 1456 total pagecache pages
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825480] 0 pages in swap cache
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825481] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825482] Free swap = 0kB
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.825483] Total swap = 0kB
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830579] 985072 pages RAM
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830582] 24998 pages reserved
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830583] 3786 pages shared
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830584] 953299 pages non-shared
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830586] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830595] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830599] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830602] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830605] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830608] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830611] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830615] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830618] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830622] [ 790] 101 790 63477 615 1 0 0 rsyslogd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830625] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830628] [ 831] 0 831 4226 41 1 0 0 atd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830631] [ 975] 0 975 6276 79 0 0 0 master
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830634] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830638] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830641] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830644] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830647] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830650] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830653] [15657] 0 15657 4777 59 0 0 0 cron
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830657] [ 654] 1012 654 70351 5746 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830660] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830663] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830666] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830669] [13468] 0 13468 89584 2524 0 0 0 php5-fpm
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830672] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830675] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830678] [ 5841] 1012 5841 68618 5646 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830681] [ 757] 108 757 6792 76 0 0 0 pickup
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830685] [13494] 108 13494 6818 81 0 0 0 cleanup
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830688] [13495] 108 13495 6795 73 0 0 0 trivial-rewrite
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830691] [13496] 108 13496 11107 157 1 0 0 smtp
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830694] [13582] 108 13582 11107 158 1 0 0 smtp
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830697] [14021] 1012 14021 647943 584764 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830700] [14319] 1012 14319 300584 237475 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830703] [14408] 1012 14408 68580 5605 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830706] [14410] 1012 14410 1099 24 0 0 0 sh
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830709] [14411] 1012 14411 68982 6015 1 0 0 php
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830712] [14439] 1012 14439 1099 26 1 0 0 sh
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830715] [14440] 1012 14440 4254 39 1 0 0 git
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830718] Out of memory: Kill process 14021 (php) score 610 or sacrifice child
Jul 14 20:48:41 ip-10-61-174-46 kernel: [1361816.830729] Killed process 14021 (php) total-vm:2591772kB, anon-rss:2339056kB, file-rss:0kB
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077037] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077041] php cpuset=/ mems_allowed=0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077044] Pid: 14408, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077045] Call Trace:
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077053] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077055] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077057] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077060] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077063] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077068] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077071] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077073] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077076] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077079] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077082] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077084] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077086] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077089] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077093] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077096] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077099] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077101] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077104] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077107] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077109] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077111] Mem-Info:
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077112] Node 0 DMA per-cpu:
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077114] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077116] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077117] Node 0 DMA32 per-cpu:
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077119] CPU 0: hi: 186, btch: 31 usd: 30
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077120] CPU 1: hi: 186, btch: 31 usd: 82
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077122] active_anon:933055 inactive_anon:1393 isolated_anon:0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077123] active_file:17 inactive_file:32 isolated_file:0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077124] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077125] free:4036 slab_reclaimable:2240 slab_unreclaimable:2637
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077125] mapped:1363 shmem:1405 pagetables:3254 bounce:0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077127] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077133] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077136] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3732220kB inactive_anon:5572kB active_file:68kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:8960kB slab_unreclaimable:10548kB kernel_stack:768kB pagetables:13016kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:197 all_unreclaimable? yes
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077143] lowmem_reserve[]: 0 0 0 0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077145] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077152] Node 0 DMA32: 241*4kB 464*8kB 68*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077159] 1437 total pagecache pages
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077160] 0 pages in swap cache
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077161] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077162] Free swap = 0kB
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.077163] Total swap = 0kB
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082359] 985072 pages RAM
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082362] 24998 pages reserved
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082363] 3789 pages shared
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082364] 953242 pages non-shared
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082366] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082375] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082378] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082382] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082385] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082388] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082391] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082395] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082398] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082402] [ 790] 101 790 63477 615 1 0 0 rsyslogd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082406] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082409] [ 831] 0 831 4226 41 1 0 0 atd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082412] [ 975] 0 975 6276 79 1 0 0 master
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082415] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082418] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082421] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082424] [ 1051] 0 1051 201626 844 0 0 0 collectd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082427] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082431] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082434] [15657] 0 15657 4777 59 0 0 0 cron
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082437] [ 654] 1012 654 70351 5746 1 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082440] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082443] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082446] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082449] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082452] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082455] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082458] [ 5841] 1012 5841 68618 5646 1 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082461] [ 757] 108 757 6792 76 1 0 0 pickup
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082465] [13494] 108 13494 6818 81 0 0 0 cleanup
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082468] [13495] 108 13495 6795 73 1 0 0 trivial-rewrite
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082471] [13496] 108 13496 11107 157 1 0 0 smtp
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082474] [13582] 108 13582 11107 158 0 0 0 smtp
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082477] [14319] 1012 14319 886155 823070 1 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082480] [14408] 1012 14408 68580 5605 0 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082483] [14441] 1012 14441 68580 5603 0 0 0 php
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082486] Out of memory: Kill process 14319 (php) score 859 or sacrifice child
Jul 14 20:48:53 ip-10-61-174-46 kernel: [1361829.082497] Killed process 14319 (php) total-vm:3544620kB, anon-rss:3292280kB, file-rss:0kB
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048666] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048670] php cpuset=/ mems_allowed=0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048672] Pid: 32277, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048674] Call Trace:
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048681] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048683] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048685] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048688] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048691] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048695] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048697] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048700] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048703] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048705] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048709] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048711] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048713] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048716] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048718] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048722] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048724] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048727] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048730] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048732] Mem-Info:
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048733] Node 0 DMA per-cpu:
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048735] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048736] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048737] Node 0 DMA32 per-cpu:
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048739] CPU 0: hi: 186, btch: 31 usd: 30
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048740] CPU 1: hi: 186, btch: 31 usd: 26
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048743] active_anon:932511 inactive_anon:1393 isolated_anon:0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048743] active_file:0 inactive_file:0 isolated_file:0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048744] unevictable:0 dirty:16 writeback:0 unstable:0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048745] free:4034 slab_reclaimable:2806 slab_unreclaimable:2724
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048745] mapped:1357 shmem:1405 pagetables:3173 bounce:0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048747] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048753] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048756] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3730044kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:64kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:11224kB slab_unreclaimable:10896kB kernel_stack:768kB pagetables:12692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:111 all_unreclaimable? yes
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048763] lowmem_reserve[]: 0 0 0 0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048765] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048772] Node 0 DMA32: 1428*4kB 2*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048779] 1439 total pagecache pages
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048780] 0 pages in swap cache
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048781] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048782] Free swap = 0kB
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.048783] Total swap = 0kB
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054023] 985072 pages RAM
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054025] 24998 pages reserved
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054026] 3783 pages shared
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054027] 953301 pages non-shared
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054029] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054038] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054042] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054045] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054048] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054051] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054054] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054058] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054061] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054065] [ 790] 101 790 63477 615 0 0 0 rsyslogd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054068] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054071] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054075] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054078] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054081] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054084] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054087] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054090] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054093] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054097] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054100] [ 654] 1012 654 70351 5748 1 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054103] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054106] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054109] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054113] [13468] 0 13468 89584 2524 1 0 0 php5-fpm
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054116] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054119] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054122] [ 5841] 1012 5841 68618 5647 1 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054125] [32277] 1012 32277 886676 823439 0 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054128] [13160] 108 13160 6792 70 1 0 0 pickup
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054131] [22662] 108 22662 6818 73 1 0 0 cleanup
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054134] [22663] 108 22663 6795 71 1 0 0 trivial-rewrite
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054137] [22664] 108 22664 11107 158 0 0 0 smtp
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054140] [22672] 108 22672 11107 159 1 0 0 smtp
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054143] [22676] 108 22676 11107 159 1 0 0 smtp
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054147] [24093] 1012 24093 73115 10147 1 0 0 php
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054149] Out of memory: Kill process 32277 (php) score 859 or sacrifice child
Jul 14 22:29:43 ip-10-61-174-46 kernel: [1367879.054161] Killed process 32277 (php) total-vm:3546704kB, anon-rss:3293756kB, file-rss:0kB
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824204] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824209] php5-fpm cpuset=/ mems_allowed=0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824213] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824215] Call Trace:
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824223] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824226] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824230] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824233] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824238] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824242] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824245] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824249] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824253] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824257] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824261] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824264] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824267] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824271] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824275] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824280] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824285] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824288] [<ffffffff8117ae2d>] ? fget+0x1d/0x90
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824292] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824294] Mem-Info:
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824295] Node 0 DMA per-cpu:
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824298] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824300] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824301] Node 0 DMA32 per-cpu:
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824304] CPU 0: hi: 186, btch: 31 usd: 112
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824305] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824309] active_anon:932989 inactive_anon:1393 isolated_anon:0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824310] active_file:0 inactive_file:27 isolated_file:0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824311] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824312] free:4048 slab_reclaimable:2489 slab_unreclaimable:2615
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824313] mapped:1352 shmem:1405 pagetables:3078 bounce:0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824315] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824323] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824327] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731956kB inactive_anon:5572kB active_file:0kB inactive_file:108kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9956kB slab_unreclaimable:10460kB kernel_stack:728kB pagetables:12312kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:295 all_unreclaimable? yes
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824335] lowmem_reserve[]: 0 0 0 0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824339] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824348] Node 0 DMA32: 802*4kB 183*8kB 55*16kB 6*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824358] 1432 total pagecache pages
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824360] 0 pages in swap cache
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824362] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824363] Free swap = 0kB
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.824364] Total swap = 0kB
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831789] 985072 pages RAM
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831791] 24998 pages reserved
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831792] 3783 pages shared
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831793] 953242 pages non-shared
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831795] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831804] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831808] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831811] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831814] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831817] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831820] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831823] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831827] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831831] [ 790] 101 790 63477 615 0 0 0 rsyslogd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831834] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831837] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831840] [ 975] 0 975 6276 80 0 0 0 master
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831843] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831846] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831849] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831852] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831855] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831859] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831862] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831865] [ 654] 1012 654 70351 5747 0 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831868] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831871] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831874] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831877] [13468] 0 13468 89584 2509 1 0 0 php5-fpm
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831880] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831883] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831886] [ 5841] 1012 5841 68618 5647 1 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831890] [13160] 108 13160 6792 76 1 0 0 pickup
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831893] [24274] 1012 24274 892212 829073 0 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831896] [31890] 1012 31890 68580 5603 1 0 0 php
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831899] Out of memory: Kill process 24274 (php) score 865 or sacrifice child
Jul 14 22:48:19 ip-10-61-174-46 kernel: [1368994.831911] Killed process 24274 (php) total-vm:3568848kB, anon-rss:3316292kB, file-rss:0kB
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636089] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636093] php cpuset=/ mems_allowed=0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636096] Pid: 32048, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636097] Call Trace:
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636105] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636107] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636109] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636112] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636117] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636120] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636123] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636126] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636129] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636131] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636134] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636137] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636139] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636142] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636145] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636147] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636149] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636152] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636155] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636157] Mem-Info:
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636158] Node 0 DMA per-cpu:
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636160] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636161] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636162] Node 0 DMA32 per-cpu:
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636164] CPU 0: hi: 186, btch: 31 usd: 0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636165] CPU 1: hi: 186, btch: 31 usd: 0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636168] active_anon:933255 inactive_anon:1393 isolated_anon:0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636168] active_file:0 inactive_file:0 isolated_file:0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636169] unevictable:0 dirty:7 writeback:0 unstable:0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636170] free:4048 slab_reclaimable:2346 slab_unreclaimable:2621
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636170] mapped:1386 shmem:1405 pagetables:3069 bounce:0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636172] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636179] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636182] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733020kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:28kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:9384kB slab_unreclaimable:10484kB kernel_stack:784kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636188] lowmem_reserve[]: 0 0 0 0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636190] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636197] Node 0 DMA32: 604*4kB 104*8kB 104*16kB 28*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636204] 1445 total pagecache pages
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636205] 0 pages in swap cache
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636207] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636208] Free swap = 0kB
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.636209] Total swap = 0kB
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641763] 985072 pages RAM
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641766] 24998 pages reserved
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641767] 3796 pages shared
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641768] 953342 pages non-shared
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641770] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641779] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641782] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641785] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641788] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641791] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641795] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641798] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641801] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641806] [ 790] 101 790 63477 615 1 0 0 rsyslogd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641809] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641812] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641815] [ 975] 0 975 6276 79 1 0 0 master
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641818] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641822] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641825] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641828] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641831] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641834] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641837] [15657] 0 15657 4777 59 1 0 0 cron
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641841] [ 654] 1012 654 70351 5751 1 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641844] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641847] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641850] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641853] [13468] 0 13468 89584 2490 1 0 0 php5-fpm
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641856] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641859] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641863] [ 5841] 1012 5841 68618 5646 1 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641866] [13160] 108 13160 6792 76 1 0 0 pickup
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641869] [32048] 1012 32048 892364 829326 0 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641872] [32111] 1012 32111 68580 5606 1 0 0 php
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641875] Out of memory: Kill process 32048 (php) score 865 or sacrifice child
Jul 14 22:48:53 ip-10-61-174-46 kernel: [1369028.641886] Killed process 32048 (php) total-vm:3569456kB, anon-rss:3317304kB, file-rss:0kB
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225034] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225039] php cpuset=/ mems_allowed=0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225042] Pid: 18795, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225043] Call Trace:
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225050] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225053] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225055] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225058] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225063] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225065] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225069] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225071] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225074] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225077] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225080] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225082] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225085] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225087] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225090] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225092] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225094] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225096] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225099] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225102] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225104] Mem-Info:
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225105] Node 0 DMA per-cpu:
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225107] CPU 0: hi: 0, btch: 1 usd: 0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225108] CPU 1: hi: 0, btch: 1 usd: 0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225109] Node 0 DMA32 per-cpu:
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225111] CPU 0: hi: 186, btch: 31 usd: 30
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225112] CPU 1: hi: 186, btch: 31 usd: 30
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225114] active_anon:932295 inactive_anon:1393 isolated_anon:0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225115] active_file:22 inactive_file:0 isolated_file:0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225116] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225116] free:4038 slab_reclaimable:2915 slab_unreclaimable:2716
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225117] mapped:1371 shmem:1405 pagetables:3303 bounce:0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225119] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225125] lowmem_reserve[]: 0 3772 3772 3772
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225128] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3729180kB inactive_anon:5572kB active_file:88kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5484kB shmem:5620kB slab_reclaimable:11660kB slab_unreclaimable:10864kB kernel_stack:784kB pagetables:13212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:264 all_unreclaimable? yes
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225135] lowmem_reserve[]: 0 0 0 0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225137] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225144] Node 0 DMA32: 1454*4kB 1*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225151] 1421 total pagecache pages
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225152] 0 pages in swap cache
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225153] Swap cache stats: add 0, delete 0, find 0/0
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225154] Free swap = 0kB
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.225155] Total swap = 0kB
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230491] 985072 pages RAM
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230493] 24998 pages reserved
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230495] 3785 pages shared
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230496] 953273 pages non-shared
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230497] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230507] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230511] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230514] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230517] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230520] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230524] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230527] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230530] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230535] [ 790] 101 790 63477 615 1 0 0 rsyslogd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230538] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230541] [ 831] 0 831 4226 41 0 0 0 atd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230544] [ 975] 0 975 6276 80 1 0 0 master
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230547] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230550] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230553] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230556] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230560] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230563] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230566] [15657] 0 15657 4777 60 0 0 0 cron
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230569] [ 654] 1012 654 70351 5750 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230572] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230575] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230578] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230582] [13468] 0 13468 89584 2527 1 0 0 php5-fpm
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230585] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230588] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230591] [ 5841] 1012 5841 68618 5647 0 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230594] [ 1165] 108 1165 6792 70 0 0 0 pickup
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230597] [18795] 1012 18795 877707 814659 0 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230600] [19509] 1012 19509 70570 7579 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230603] [19625] 1012 19625 1099 25 1 0 0 sh
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230607] [19626] 1012 19626 68825 5871 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230610] [19632] 1012 19632 1099 25 1 0 0 sh
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230613] [19633] 1012 19633 4700 74 1 0 0 git
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230616] [19634] 1012 19634 68499 5552 1 0 0 php
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230619] [19636] 1012 19636 1099 25 1 0 0 sh
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230622] [19637] 1012 19637 10901 176 1 0 0 ssh
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230625] Out of memory: Kill process 18795 (php) score 850 or sacrifice child
Jul 14 23:33:30 ip-10-61-174-46 kernel: [1371706.230636] Killed process 18795 (php) total-vm:3510828kB, anon-rss:3258636kB, file-rss:0kB
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231093] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231098] php cpuset=/ mems_allowed=0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231102] Pid: 24206, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231104] Call Trace:
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231113] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231116] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231119] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231124] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231128] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231133] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231139] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231142] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231146] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231149] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231153] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231157] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231160] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231164] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231169] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231172] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231176] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231180] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231183] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231186] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231190] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231192] Mem-Info:
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231193] Node 0 DMA per-cpu:
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231196] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231198] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231200] Node 0 DMA32 per-cpu:
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231201] CPU 0: hi: 186, btch: 31 usd: 121
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231203] CPU 1: hi: 186, btch: 31 usd: 26
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231205] active_anon:932109 inactive_anon:1393 isolated_anon:0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231206] active_file:4 inactive_file:19 isolated_file:21
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231207] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231207] free:4043 slab_reclaimable:2896 slab_unreclaimable:2700
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231208] mapped:1396 shmem:1405 pagetables:3354 bounce:0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231210] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231216] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231219] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3728436kB inactive_anon:5572kB active_file:16kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):84kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5584kB shmem:5620kB slab_reclaimable:11584kB slab_unreclaimable:10800kB kernel_stack:808kB pagetables:13416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:235 all_unreclaimable? yes
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231226] lowmem_reserve[]: 0 0 0 0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231229] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231235] Node 0 DMA32: 1455*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231242] 1454 total pagecache pages
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231243] 0 pages in swap cache
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231244] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231245] Free swap = 0kB
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.231246] Total swap = 0kB
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236409] 985072 pages RAM
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236411] 24998 pages reserved
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236413] 3794 pages shared
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236414] 953183 pages non-shared
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236416] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236425] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236429] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236432] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236435] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236439] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236442] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236445] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236449] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236452] [ 790] 101 790 63477 620 0 0 0 rsyslogd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236456] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236459] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236462] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236465] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236468] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236472] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236475] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236478] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236481] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236485] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236488] [ 654] 1012 654 70351 5750 1 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236491] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236494] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236497] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236500] [13468] 0 13468 89584 2499 0 0 0 php5-fpm
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236503] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236506] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236509] [ 5841] 1012 5841 68618 5647 1 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236512] [19719] 1012 19719 886446 823222 0 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236516] [14792] 108 14792 6792 69 0 0 0 pickup
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236519] [23944] 108 23944 6818 74 0 0 0 cleanup
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236522] [23945] 108 23945 6795 70 1 0 0 trivial-rewrite
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236525] [23946] 108 23946 11107 158 0 0 0 smtp
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236528] [23950] 108 23950 11107 157 1 0 0 smtp
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236531] [23954] 108 23954 11107 158 0 0 0 smtp
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236534] [23958] 108 23958 11107 158 1 0 0 smtp
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236537] [23962] 108 23962 11107 159 1 0 0 smtp
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236540] [23966] 108 23966 6792 70 1 0 0 scache
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236543] [24206] 1012 24206 68580 5605 1 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236546] [24343] 1012 24343 1099 25 1 0 0 sh
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236550] [24344] 1012 24344 66354 3941 0 0 0 php
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236552] Out of memory: Kill process 19719 (php) score 859 or sacrifice child
Jul 15 00:21:50 ip-10-61-174-46 kernel: [1374606.236564] Killed process 19719 (php) total-vm:3545784kB, anon-rss:3292888kB, file-rss:0kB
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260107] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260111] php cpuset=/ mems_allowed=0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260114] Pid: 24470, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260116] Call Trace:
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260122] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260124] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260126] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260129] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260134] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260137] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260140] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260143] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260146] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260150] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260152] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260155] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260157] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260160] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260163] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260165] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260167] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260169] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260171] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260174] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260177] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260178] Mem-Info:
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260180] Node 0 DMA per-cpu:
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260181] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260183] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260184] Node 0 DMA32 per-cpu:
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260185] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260187] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260189] active_anon:932884 inactive_anon:1393 isolated_anon:0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260190] active_file:0 inactive_file:0 isolated_file:0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260191] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260191] free:4048 slab_reclaimable:2704 slab_unreclaimable:2611
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260192] mapped:1321 shmem:1405 pagetables:3076 bounce:0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260193] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260200] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260203] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731536kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5284kB shmem:5620kB slab_reclaimable:10816kB slab_unreclaimable:10444kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260209] lowmem_reserve[]: 0 0 0 0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260212] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260218] Node 0 DMA32: 1222*4kB 107*8kB 2*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260225] 1461 total pagecache pages
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260226] 0 pages in swap cache
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260227] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260229] Free swap = 0kB
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.260229] Total swap = 0kB
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265385] 985072 pages RAM
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265388] 24998 pages reserved
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265389] 3802 pages shared
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265390] 953335 pages non-shared
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265392] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265401] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265404] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265408] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265411] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265414] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265417] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265420] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265424] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265428] [ 790] 101 790 63477 620 0 0 0 rsyslogd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265431] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265434] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265437] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265440] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265443] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265446] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265449] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265453] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265456] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265459] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265462] [ 654] 1012 654 70351 5750 1 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265465] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265468] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265471] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265474] [13468] 0 13468 89584 2536 1 0 0 php5-fpm
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265477] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265480] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265483] [ 5841] 1012 5841 68618 5647 1 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265487] [14792] 108 14792 6792 75 1 0 0 pickup
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265490] [24470] 1012 24470 892106 828963 0 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265493] [31276] 1012 31276 68580 5604 1 0 0 php
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265496] Out of memory: Kill process 24470 (php) score 865 or sacrifice child
Jul 15 00:29:46 ip-10-61-174-46 kernel: [1375082.265507] Killed process 24470 (php) total-vm:3568424kB, anon-rss:3315852kB, file-rss:0kB
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240238] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240245] php cpuset=/ mems_allowed=0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240248] Pid: 15998, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240250] Call Trace:
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240257] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240259] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240261] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240264] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240269] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240272] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240275] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240277] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240280] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240283] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240286] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240289] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240291] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240294] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240296] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240299] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240301] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240303] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240305] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240308] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240309] Mem-Info:
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240311] Node 0 DMA per-cpu:
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240313] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240314] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240315] Node 0 DMA32 per-cpu:
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240317] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240318] CPU 1: hi: 186, btch: 31 usd: 29
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240321] active_anon:932821 inactive_anon:1393 isolated_anon:0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240321] active_file:0 inactive_file:42 isolated_file:0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240322] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240323] free:4019 slab_reclaimable:2807 slab_unreclaimable:2638
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240323] mapped:1360 shmem:1405 pagetables:2979 bounce:0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240325] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240331] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240334] Node 0 DMA32 free:7740kB min:7848kB low:9808kB high:11772kB active_anon:3731284kB inactive_anon:5572kB active_file:0kB inactive_file:168kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:11228kB slab_unreclaimable:10552kB kernel_stack:728kB pagetables:11916kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240340] lowmem_reserve[]: 0 0 0 0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240343] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240350] Node 0 DMA32: 1411*4kB 4*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7740kB
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240356] 1497 total pagecache pages
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240358] 0 pages in swap cache
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240359] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240360] Free swap = 0kB
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.240361] Total swap = 0kB
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245603] 985072 pages RAM
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245605] 24998 pages reserved
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245606] 3802 pages shared
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245608] 953306 pages non-shared
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245609] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245618] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245622] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245625] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245628] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245631] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245635] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245638] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245641] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245645] [ 790] 101 790 63477 617 0 0 0 rsyslogd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245648] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245651] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245655] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245657] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245660] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245664] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245667] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245670] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245673] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245676] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245680] [ 654] 1012 654 70351 5757 1 0 0 php
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245683] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245686] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245689] [ 678] 1012 678 70152 5567 1 0 0 php
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245692] [13468] 0 13468 89584 2521 1 0 0 php5-fpm
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245695] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245698] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245701] [ 5841] 1012 5841 68618 5649 1 0 0 php
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245704] [14792] 108 14792 6792 75 0 0 0 pickup
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245707] [15998] 1012 15998 897548 834502 0 0 0 php
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245710] Out of memory: Kill process 15998 (php) score 870 or sacrifice child
Jul 15 00:48:47 ip-10-61-174-46 kernel: [1376223.245721] Killed process 15998 (php) total-vm:3590192kB, anon-rss:3338008kB, file-rss:0kB
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162741] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162746] php cpuset=/ mems_allowed=0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162749] Pid: 31567, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162751] Call Trace:
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162759] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162762] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162765] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162769] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162773] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162778] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162783] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162786] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162789] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162793] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162796] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162799] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162802] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162806] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162811] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162814] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162818] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162821] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162825] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162828] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162832] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162834] Mem-Info:
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162835] Node 0 DMA per-cpu:
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162838] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162840] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162841] Node 0 DMA32 per-cpu:
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162844] CPU 0: hi: 186, btch: 31 usd: 82
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162845] CPU 1: hi: 186, btch: 31 usd: 30
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162849] active_anon:932069 inactive_anon:1393 isolated_anon:0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162850] active_file:14 inactive_file:3 isolated_file:25
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162851] unevictable:0 dirty:2 writeback:0 unstable:0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162852] free:4043 slab_reclaimable:2858 slab_unreclaimable:2732
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162853] mapped:1402 shmem:1405 pagetables:3385 bounce:0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162855] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162863] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162867] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3728276kB inactive_anon:5572kB active_file:56kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):100kB present:3862656kB mlocked:0kB dirty:8kB writeback:0kB mapped:5608kB shmem:5620kB slab_reclaimable:11432kB slab_unreclaimable:10928kB kernel_stack:816kB pagetables:13540kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:208 all_unreclaimable? yes
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162875] lowmem_reserve[]: 0 0 0 0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162878] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162885] Node 0 DMA32: 1447*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162891] 1436 total pagecache pages
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162893] 0 pages in swap cache
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162894] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162895] Free swap = 0kB
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.162896] Total swap = 0kB
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168057] 985072 pages RAM
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168059] 24998 pages reserved
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168060] 3817 pages shared
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168061] 953215 pages non-shared
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168063] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168072] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168075] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168079] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168082] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168085] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168088] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168092] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168095] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168099] [ 790] 101 790 63477 617 0 0 0 rsyslogd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168102] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168105] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168108] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168111] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168114] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168118] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168121] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168124] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168127] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168130] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168134] [ 654] 1012 654 70351 5757 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168137] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168140] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168143] [ 678] 1012 678 70152 5567 0 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168146] [13468] 0 13468 89584 2506 1 0 0 php5-fpm
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168149] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168152] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168155] [21570] 1012 21570 884073 820966 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168158] [23281] 108 23281 6792 70 1 0 0 pickup
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168161] [ 2686] 1012 2686 68618 5646 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168165] [30252] 108 30252 6818 73 0 0 0 cleanup
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168168] [30253] 108 30253 6795 71 0 0 0 trivial-rewrite
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168171] [30254] 108 30254 11107 159 1 0 0 smtp
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168174] [30258] 108 30258 11107 158 0 0 0 smtp
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168177] [30262] 108 30262 11107 158 1 0 0 smtp
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168180] [31567] 1012 31567 68580 5604 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168183] [31837] 1012 31837 1099 24 1 0 0 sh
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168186] [31839] 1012 31839 68825 5868 1 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168189] [31851] 1012 31851 1099 25 1 0 0 sh
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168192] [31852] 1012 31852 4655 58 1 0 0 git
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168195] [31853] 1012 31853 43811 625 0 0 0 php
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168198] Out of memory: Kill process 21570 (php) score 856 or sacrifice child
Jul 15 01:30:04 ip-10-61-174-46 kernel: [1378700.168209] Killed process 21570 (php) total-vm:3536292kB, anon-rss:3283760kB, file-rss:104kB
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424144] php cpuset=/ mems_allowed=0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424147] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424149] Call Trace:
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424169] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424173] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424175] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424180] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424184] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424186] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424189] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424191] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424194] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424197] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424201] [<ffffffff811798c0>] ? vfs_write+0x110/0x180
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424204] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424205] Mem-Info:
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424207] Node 0 DMA per-cpu:
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424209] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424210] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424211] Node 0 DMA32 per-cpu:
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424213] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424214] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424216] active_anon:932401 inactive_anon:1393 isolated_anon:0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424217] active_file:34 inactive_file:40 isolated_file:0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424218] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424218] free:4048 slab_reclaimable:2988 slab_unreclaimable:2679
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424219] mapped:1381 shmem:1405 pagetables:3143 bounce:0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424221] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424227] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424230] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729604kB inactive_anon:5572kB active_file:136kB inactive_file:160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:11952kB slab_unreclaimable:10716kB kernel_stack:760kB pagetables:12572kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:28808 all_unreclaimable? yes
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424237] lowmem_reserve[]: 0 0 0 0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424239] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424246] Node 0 DMA32: 1454*4kB 14*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7992kB
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424253] 1484 total pagecache pages
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424254] 0 pages in swap cache
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424255] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424256] Free swap = 0kB
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.424257] Total swap = 0kB
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429439] 985072 pages RAM
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429441] 24998 pages reserved
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429442] 3824 pages shared
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429443] 953277 pages non-shared
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429445] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429454] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429458] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429461] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429464] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429467] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429471] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429474] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429478] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429482] [ 790] 101 790 63477 620 1 0 0 rsyslogd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429485] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429488] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429491] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429495] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429498] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429501] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429504] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429507] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429510] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429514] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429517] [ 654] 1012 654 70351 5758 1 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429520] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429523] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429526] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429529] [13468] 0 13468 89584 2491 1 0 0 php5-fpm
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429532] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429535] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429538] [ 2686] 1012 2686 68618 5646 0 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429542] [11415] 1012 11415 891137 828027 1 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429545] [19313] 108 19313 6792 70 0 0 0 pickup
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429548] [23459] 108 23459 6818 73 0 0 0 cleanup
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429551] [23460] 108 23460 6795 71 0 0 0 trivial-rewrite
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429555] [23461] 108 23461 11107 143 1 0 0 smtp
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429558] [23465] 108 23465 11107 158 1 0 0 smtp
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429561] [24144] 1012 24144 68580 5605 1 0 0 php
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429564] Out of memory: Kill process 11415 (php) score 864 or sacrifice child
Jul 15 02:21:53 ip-10-61-174-46 kernel: [1381809.429575] Killed process 11415 (php) total-vm:3564548kB, anon-rss:3312084kB, file-rss:24kB
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336304] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336308] php cpuset=/ mems_allowed=0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336311] Pid: 24495, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336313] Call Trace:
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336320] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336322] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336324] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336327] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336330] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336334] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336336] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336339] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336342] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336344] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336347] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336349] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336352] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336355] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336357] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336360] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336363] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336366] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336368] Mem-Info:
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336369] Node 0 DMA per-cpu:
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336371] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336372] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336373] Node 0 DMA32 per-cpu:
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336375] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336376] CPU 1: hi: 186, btch: 31 usd: 25
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336379] active_anon:932097 inactive_anon:1393 isolated_anon:0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336379] active_file:0 inactive_file:13 isolated_file:0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336380] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336381] free:4021 slab_reclaimable:2824 slab_unreclaimable:2731
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336381] mapped:1385 shmem:1405 pagetables:3444 bounce:0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336383] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336389] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336392] Node 0 DMA32 free:7748kB min:7848kB low:9808kB high:11772kB active_anon:3728388kB inactive_anon:5572kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5540kB shmem:5620kB slab_reclaimable:11296kB slab_unreclaimable:10924kB kernel_stack:840kB pagetables:13776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:205 all_unreclaimable? yes
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336399] lowmem_reserve[]: 0 0 0 0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336401] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336408] Node 0 DMA32: 1356*4kB 42*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336415] 1463 total pagecache pages
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336416] 0 pages in swap cache
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336417] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336418] Free swap = 0kB
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.336419] Total swap = 0kB
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341656] 985072 pages RAM
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341659] 24998 pages reserved
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341660] 3794 pages shared
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341661] 953278 pages non-shared
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341663] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341672] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341675] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341678] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341681] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341685] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341688] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341691] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341695] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341699] [ 790] 101 790 63477 623 0 0 0 rsyslogd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341702] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341705] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341709] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341712] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341715] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341718] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341721] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341724] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341728] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341731] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341734] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341737] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341740] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341743] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341746] [13468] 0 13468 89584 2531 0 0 0 php5-fpm
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341749] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341752] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341756] [ 2686] 1012 2686 68618 5646 1 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341759] [19313] 108 19313 6792 76 0 0 0 pickup
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341762] [24495] 1012 24495 878725 815583 0 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341765] [16057] 108 16057 6795 71 0 0 0 trivial-rewrite
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341768] [16066] 108 16066 6792 73 0 0 0 scache
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341771] [17771] 108 17771 6818 73 1 0 0 cleanup
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341774] [17772] 108 17772 11107 158 0 0 0 smtp
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341777] [17776] 108 17776 11107 159 1 0 0 smtp
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341781] [17780] 108 17780 11107 159 1 0 0 smtp
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341784] [17784] 108 17784 11107 159 0 0 0 smtp
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341786] [19044] 1012 19044 68580 5603 1 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341790] [19260] 1012 19260 1099 25 1 0 0 sh
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341793] [19261] 1012 19261 68825 5871 1 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341795] [19272] 1012 19272 1099 25 1 0 0 sh
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341799] [19274] 1012 19274 4651 62 0 0 0 git
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341801] [19276] 1012 19276 68499 5552 1 0 0 php
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341804] [19280] 1012 19280 1099 24 1 0 0 sh
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341807] [19282] 1012 19282 10901 176 1 0 0 ssh
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341810] Out of memory: Kill process 24495 (php) score 851 or sacrifice child
Jul 15 02:48:20 ip-10-61-174-46 kernel: [1383396.341820] Killed process 24495 (php) total-vm:3514900kB, anon-rss:3262332kB, file-rss:0kB
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437353] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437358] php5-fpm cpuset=/ mems_allowed=0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437361] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437363] Call Trace:
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437372] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437375] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437377] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437380] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437383] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437386] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437388] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437391] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437395] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437398] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437400] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437403] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437406] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437408] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437412] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437415] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437419] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437422] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437425] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437427] Mem-Info:
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437428] Node 0 DMA per-cpu:
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437430] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437431] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437433] Node 0 DMA32 per-cpu:
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437434] CPU 0: hi: 186, btch: 31 usd: 38
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437436] CPU 1: hi: 186, btch: 31 usd: 99
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437438] active_anon:931603 inactive_anon:1393 isolated_anon:0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437439] active_file:0 inactive_file:0 isolated_file:20
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437440] unevictable:0 dirty:47 writeback:0 unstable:0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437440] free:4048 slab_reclaimable:2644 slab_unreclaimable:2773
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437441] mapped:1373 shmem:1405 pagetables:3872 bounce:0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437443] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437449] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437452] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3726412kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:188kB writeback:0kB mapped:5492kB shmem:5620kB slab_reclaimable:10576kB slab_unreclaimable:11092kB kernel_stack:896kB pagetables:15488kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:206 all_unreclaimable? yes
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437459] lowmem_reserve[]: 0 0 0 0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437461] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437468] Node 0 DMA32: 514*4kB 169*8kB 23*16kB 20*32kB 14*64kB 4*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437475] 1437 total pagecache pages
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437476] 0 pages in swap cache
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437477] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437479] Free swap = 0kB
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.437479] Total swap = 0kB
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442619] 985072 pages RAM
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442622] 24998 pages reserved
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442623] 3783 pages shared
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442624] 953191 pages non-shared
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442626] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442635] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442638] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442642] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442645] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442648] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442651] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442654] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442658] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442662] [ 790] 101 790 63477 623 1 0 0 rsyslogd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442665] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442668] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442671] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442675] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442678] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442681] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442684] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442687] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442690] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442693] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442697] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442700] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442703] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442706] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442709] [13468] 0 13468 89584 2511 0 0 0 php5-fpm
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442712] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442715] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442719] [ 2686] 1012 2686 68618 5645 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442722] [19313] 108 19313 6792 76 0 0 0 pickup
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442725] [16057] 108 16057 6795 71 0 0 0 trivial-rewrite
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442728] [17771] 108 17771 6818 73 1 0 0 cleanup
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442732] [17772] 108 17772 11107 158 0 0 0 smtp
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442735] [17776] 108 17776 11107 159 1 0 0 smtp
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442738] [17780] 108 17780 11107 159 1 0 0 smtp
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442741] [17784] 108 17784 11107 159 0 0 0 smtp
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442744] [19323] 1012 19323 664126 601020 0 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442747] [19396] 1012 19396 263044 199931 0 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442750] [19619] 1012 19619 72647 9704 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442753] [19631] 1012 19631 1099 25 0 0 0 sh
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442756] [19632] 1012 19632 68825 5873 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442759] [19633] 1012 19633 1099 25 0 0 0 sh
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442762] [19634] 1012 19634 68825 5872 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442765] [19635] 1012 19635 1099 25 0 0 0 sh
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442768] [19637] 1012 19637 68825 5871 0 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442771] [19644] 1012 19644 1099 26 0 0 0 sh
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442774] [19645] 1012 19645 4655 58 0 0 0 git
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442777] [19646] 1012 19646 65507 3090 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442780] [19655] 1012 19655 1099 26 1 0 0 sh
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442783] [19656] 1012 19656 14040 63 1 0 0 git
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442785] [19658] 1012 19658 31450 417 1 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442789] [19659] 1012 19659 1099 25 1 0 0 sh
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442792] [19660] 1012 19660 4649 58 0 0 0 git
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442795] [19661] 1012 19661 23521 347 0 0 0 php
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442797] Out of memory: Kill process 19323 (php) score 627 or sacrifice child
Jul 15 02:48:44 ip-10-61-174-46 kernel: [1383419.442808] Killed process 19323 (php) total-vm:2656504kB, anon-rss:2404080kB, file-rss:0kB
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856151] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856155] php cpuset=/ mems_allowed=0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856159] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856161] Call Trace:
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856168] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856172] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856175] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856179] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856184] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856188] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856193] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856196] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856200] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856204] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856207] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856211] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856214] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856217] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856221] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856225] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856229] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856232] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856235] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856237] Mem-Info:
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856238] Node 0 DMA per-cpu:
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856240] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856242] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856243] Node 0 DMA32 per-cpu:
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856244] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856245] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856248] active_anon:932629 inactive_anon:1393 isolated_anon:0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856249] active_file:9 inactive_file:11 isolated_file:0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856249] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856250] free:4048 slab_reclaimable:2581 slab_unreclaimable:2674
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856251] mapped:1354 shmem:1405 pagetables:3305 bounce:0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856252] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856259] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856262] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730516kB inactive_anon:5572kB active_file:36kB inactive_file:44kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:10324kB slab_unreclaimable:10696kB kernel_stack:784kB pagetables:13220kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8675 all_unreclaimable? yes
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856268] lowmem_reserve[]: 0 0 0 0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856271] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856277] Node 0 DMA32: 756*4kB 272*8kB 40*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856284] 1426 total pagecache pages
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856285] 0 pages in swap cache
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856287] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856288] Free swap = 0kB
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.856289] Total swap = 0kB
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861464] 985072 pages RAM
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861466] 24998 pages reserved
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861467] 3801 pages shared
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861468] 953289 pages non-shared
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861471] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861479] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861482] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861485] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861488] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861492] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861495] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861498] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861501] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861505] [ 790] 101 790 63477 623 1 0 0 rsyslogd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861509] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861512] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861515] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861518] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861521] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861524] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861527] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861530] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861534] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861537] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861540] [ 654] 1012 654 70351 5762 1 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861543] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861546] [ 658] 1012 658 134393 70798 0 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861549] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861552] [13468] 0 13468 89584 2502 0 0 0 php5-fpm
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861555] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861558] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861561] [ 2686] 1012 2686 68618 5645 1 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861565] [19313] 108 19313 6792 76 0 0 0 pickup
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861568] [16057] 108 16057 6795 75 1 0 0 trivial-rewrite
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861571] [17771] 108 17771 6818 81 1 0 0 cleanup
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861574] [17772] 108 17772 11107 157 1 0 0 smtp
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861577] [17776] 108 17776 11107 158 0 0 0 smtp
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861580] [17780] 108 17780 11107 158 1 0 0 smtp
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861583] [17784] 108 17784 11107 159 0 0 0 smtp
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861587] [19396] 1012 19396 875562 812467 0 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861590] [19619] 1012 19619 73702 10752 1 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861593] [19761] 1012 19761 73267 10297 1 0 0 php
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861595] Out of memory: Kill process 19396 (php) score 848 or sacrifice child
Jul 15 02:48:59 ip-10-61-174-46 kernel: [1383434.861607] Killed process 19396 (php) total-vm:3502248kB, anon-rss:3249868kB, file-rss:0kB
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496281] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496285] php cpuset=/ mems_allowed=0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496288] Pid: 3247, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496289] Call Trace:
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496296] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496299] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496301] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496304] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496307] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496310] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496312] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496315] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496318] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496321] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496323] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496326] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496329] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496331] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496334] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496336] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496338] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496341] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496344] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496345] Mem-Info:
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496347] Node 0 DMA per-cpu:
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496349] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496350] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496351] Node 0 DMA32 per-cpu:
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496353] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496354] CPU 1: hi: 186, btch: 31 usd: 31
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496356] active_anon:932673 inactive_anon:1393 isolated_anon:0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496357] active_file:0 inactive_file:0 isolated_file:0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496358] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496358] free:4044 slab_reclaimable:2874 slab_unreclaimable:2641
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496359] mapped:1363 shmem:1405 pagetables:3073 bounce:0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496361] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496367] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496370] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3730692kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:11496kB slab_unreclaimable:10564kB kernel_stack:768kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:75 all_unreclaimable? yes
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496377] lowmem_reserve[]: 0 0 0 0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496379] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496386] Node 0 DMA32: 1434*4kB 7*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496392] 1429 total pagecache pages
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496394] 0 pages in swap cache
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496395] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496396] Free swap = 0kB
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.496397] Total swap = 0kB
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501595] 985072 pages RAM
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501597] 24998 pages reserved
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501599] 3784 pages shared
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501600] 953292 pages non-shared
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501602] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501610] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501614] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501617] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501620] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501623] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501627] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501630] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501633] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501638] [ 790] 101 790 63477 623 1 0 0 rsyslogd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501641] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501644] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501647] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501650] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501653] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501657] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501660] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501663] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501666] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501669] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501672] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501675] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501679] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501682] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501685] [13468] 0 13468 89584 2526 1 0 0 php5-fpm
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501688] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501691] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501694] [ 2686] 1012 2686 68618 5646 1 0 0 php
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501697] [30422] 108 30422 6792 70 0 0 0 pickup
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501700] [ 3247] 1012 3247 896947 833822 0 0 0 php
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501704] [24042] 108 24042 6792 70 0 0 0 scache
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501707] [26329] 108 26329 6818 72 0 0 0 cleanup
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501710] [26330] 108 26330 6795 69 1 0 0 trivial-rewrite
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501713] [26331] 108 26331 11107 158 1 0 0 smtp
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501716] [26335] 108 26335 11107 158 1 0 0 smtp
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501719] Out of memory: Kill process 3247 (php) score 870 or sacrifice child
Jul 15 03:30:05 ip-10-61-174-46 kernel: [1385901.501729] Killed process 3247 (php) total-vm:3587788kB, anon-rss:3335288kB, file-rss:0kB
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824128] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824132] php cpuset=/ mems_allowed=0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824136] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824138] Call Trace:
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824145] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824148] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824151] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824155] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824161] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824165] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824169] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824172] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824175] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824179] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824182] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824185] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824193] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824196] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824198] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824200] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824204] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824207] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824209] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824211] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824213] Mem-Info:
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824214] Node 0 DMA per-cpu:
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824216] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824218] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824219] Node 0 DMA32 per-cpu:
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824220] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824222] CPU 1: hi: 186, btch: 31 usd: 1
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824224] active_anon:932702 inactive_anon:1393 isolated_anon:0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824225] active_file:15 inactive_file:18 isolated_file:8
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824226] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824226] free:4047 slab_reclaimable:2694 slab_unreclaimable:2644
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824227] mapped:1356 shmem:1405 pagetables:3170 bounce:0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824235] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824238] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3730808kB inactive_anon:5572kB active_file:60kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10776kB slab_unreclaimable:10576kB kernel_stack:760kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:932 all_unreclaimable? yes
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824244] lowmem_reserve[]: 0 0 0 0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824247] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824253] Node 0 DMA32: 1479*4kB 5*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824260] 1439 total pagecache pages
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824261] 0 pages in swap cache
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824262] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824264] Free swap = 0kB
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.824264] Total swap = 0kB
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829519] 985072 pages RAM
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829522] 24998 pages reserved
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829523] 3778 pages shared
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829525] 953317 pages non-shared
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829526] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829534] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829538] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829541] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829544] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829547] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829551] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829554] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829557] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829562] [ 790] 101 790 63477 623 0 0 0 rsyslogd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829565] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829568] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829571] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829574] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829577] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829580] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829584] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829587] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829590] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829593] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829596] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829599] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829603] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829606] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829609] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829612] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829615] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829618] [ 2686] 1012 2686 68618 5647 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829621] [29283] 108 29283 6792 69 1 0 0 pickup
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829624] [ 9277] 1012 9277 891230 828089 0 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829628] [19999] 1012 19999 68580 5604 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829631] [20179] 1012 20179 1099 24 1 0 0 sh
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829634] [20180] 1012 20180 28546 368 0 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829637] [20181] 1012 20181 1099 25 1 0 0 sh
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829640] [20182] 1012 20182 12611 262 1 0 0 php
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829642] Out of memory: Kill process 9277 (php) score 864 or sacrifice child
Jul 15 04:29:49 ip-10-61-174-46 kernel: [1389484.829653] Killed process 9277 (php) total-vm:3564920kB, anon-rss:3312356kB, file-rss:0kB
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288402] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288406] php cpuset=/ mems_allowed=0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288409] Pid: 3903, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288410] Call Trace:
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288418] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288420] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288422] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288425] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288428] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288432] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288434] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288437] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288440] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288443] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288445] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288448] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288451] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288453] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288455] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288458] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288461] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288463] Mem-Info:
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288464] Node 0 DMA per-cpu:
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288466] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288467] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288468] Node 0 DMA32 per-cpu:
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288470] CPU 0: hi: 186, btch: 31 usd: 13
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288471] CPU 1: hi: 186, btch: 31 usd: 54
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288474] active_anon:932501 inactive_anon:1393 isolated_anon:0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288474] active_file:0 inactive_file:53 isolated_file:0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288475] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288476] free:4018 slab_reclaimable:2626 slab_unreclaimable:2649
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288476] mapped:1381 shmem:1405 pagetables:3354 bounce:0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288478] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288484] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288487] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3730004kB inactive_anon:5572kB active_file:0kB inactive_file:212kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:10504kB slab_unreclaimable:10596kB kernel_stack:808kB pagetables:13416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:419 all_unreclaimable? yes
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288494] lowmem_reserve[]: 0 0 0 0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288496] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288503] Node 0 DMA32: 1308*4kB 57*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288510] 1438 total pagecache pages
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288511] 0 pages in swap cache
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288512] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288513] Free swap = 0kB
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.288514] Total swap = 0kB
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294147] 985072 pages RAM
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294150] 24998 pages reserved
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294151] 3775 pages shared
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294152] 953315 pages non-shared
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294154] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294163] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294166] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294170] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294173] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294176] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294179] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294182] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294186] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294190] [ 790] 101 790 63477 623 1 0 0 rsyslogd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294193] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294196] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294199] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294202] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294205] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294208] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294212] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294215] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294218] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294221] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294224] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294227] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294230] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294234] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294237] [13468] 0 13468 89584 2536 0 0 0 php5-fpm
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294240] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294243] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294246] [ 2686] 1012 2686 68618 5647 1 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294249] [ 3903] 1012 3903 886275 823228 0 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294252] [ 4486] 108 4486 6818 74 0 0 0 cleanup
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294255] [ 4487] 108 4487 6795 70 1 0 0 trivial-rewrite
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294258] [ 4488] 108 4488 11107 158 0 0 0 smtp
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294262] [ 4492] 108 4492 11107 159 1 0 0 smtp
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294265] [ 4496] 108 4496 11107 158 0 0 0 smtp
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294268] [ 4501] 108 4501 11107 142 0 0 0 smtp
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294271] [ 4505] 108 4505 6792 70 0 0 0 pickup
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294274] [ 4506] 108 4506 11107 159 0 0 0 smtp
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294277] [ 4510] 108 4510 6792 70 1 0 0 scache
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294280] [ 5491] 1012 5491 68580 5604 0 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294283] [ 5725] 1012 5725 1099 25 1 0 0 sh
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294286] [ 5726] 1012 5726 67296 4334 1 0 0 php
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294289] Out of memory: Kill process 3903 (php) score 859 or sacrifice child
Jul 15 04:47:37 ip-10-61-174-46 kernel: [1390553.294299] Killed process 3903 (php) total-vm:3545100kB, anon-rss:3292912kB, file-rss:0kB
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924413] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924417] php cpuset=/ mems_allowed=0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924420] Pid: 5880, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924421] Call Trace:
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924429] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924431] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924433] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924436] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924439] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924443] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924445] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924448] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924452] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924455] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924458] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924460] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924463] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924465] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924468] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924470] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924472] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924474] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924477] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924479] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924481] Mem-Info:
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924482] Node 0 DMA per-cpu:
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924484] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924486] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924487] Node 0 DMA32 per-cpu:
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924488] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924490] CPU 1: hi: 186, btch: 31 usd: 58
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924492] active_anon:932901 inactive_anon:1393 isolated_anon:0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924493] active_file:0 inactive_file:0 isolated_file:0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924494] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924494] free:4044 slab_reclaimable:2363 slab_unreclaimable:2633
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924495] mapped:1361 shmem:1405 pagetables:3193 bounce:0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924496] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924503] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924506] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731604kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:9452kB slab_unreclaimable:10532kB kernel_stack:752kB pagetables:12772kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:72 all_unreclaimable? yes
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924512] lowmem_reserve[]: 0 0 0 0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924515] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924522] Node 0 DMA32: 824*4kB 222*8kB 39*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924528] 1416 total pagecache pages
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924530] 0 pages in swap cache
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924531] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924532] Free swap = 0kB
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.924533] Total swap = 0kB
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929646] 985072 pages RAM
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929649] 24998 pages reserved
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929650] 3792 pages shared
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929651] 953258 pages non-shared
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929653] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929662] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929665] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929669] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929672] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929675] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929678] [ 664] 0 664 1815 123 0 0 0 dhclient3
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929682] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929685] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929690] [ 790] 101 790 63477 623 1 0 0 rsyslogd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929693] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929696] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929699] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929702] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929706] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929709] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929712] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929715] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929718] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929721] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929725] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929728] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929731] [ 658] 1012 658 134393 70798 1 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929734] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929737] [13468] 0 13468 89584 2503 0 0 0 php5-fpm
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929740] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929743] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929746] [ 2686] 1012 2686 68618 5646 1 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929750] [ 4505] 108 4505 6792 70 0 0 0 pickup
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929753] [ 4510] 108 4510 6792 70 1 0 0 scache
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929756] [ 5880] 1012 5880 886027 822972 0 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929759] [ 6615] 1012 6615 68580 5604 1 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929762] [ 6802] 1012 6802 1099 25 0 0 0 sh
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929765] [ 6803] 1012 6803 68913 5897 1 0 0 php
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929768] Out of memory: Kill process 5880 (php) score 858 or sacrifice child
Jul 15 04:48:49 ip-10-61-174-46 kernel: [1390624.929780] Killed process 5880 (php) total-vm:3544108kB, anon-rss:3291888kB, file-rss:0kB
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412124] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412128] php cpuset=/ mems_allowed=0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412131] Pid: 4819, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412132] Call Trace:
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412139] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412141] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412143] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412146] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412151] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412153] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412157] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412159] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412162] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412164] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412167] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412169] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412172] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412175] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412177] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412180] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412182] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412185] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412187] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412191] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412194] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412197] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412199] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412201] Mem-Info:
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412202] Node 0 DMA per-cpu:
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412204] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412205] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412207] Node 0 DMA32 per-cpu:
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412208] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412209] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412212] active_anon:932627 inactive_anon:1393 isolated_anon:0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412213] active_file:14 inactive_file:14 isolated_file:0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412213] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412214] free:4048 slab_reclaimable:2821 slab_unreclaimable:2630
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412215] mapped:1368 shmem:1405 pagetables:3155 bounce:0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412216] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412223] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730508kB inactive_anon:5572kB active_file:56kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:11284kB slab_unreclaimable:10520kB kernel_stack:760kB pagetables:12620kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15932 all_unreclaimable? yes
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412232] lowmem_reserve[]: 0 0 0 0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412234] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412241] Node 0 DMA32: 1452*4kB 6*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7936kB
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412248] 1457 total pagecache pages
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412249] 0 pages in swap cache
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412250] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412251] Free swap = 0kB
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.412252] Total swap = 0kB
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417466] 985072 pages RAM
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417468] 24998 pages reserved
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417469] 3806 pages shared
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417470] 953307 pages non-shared
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417472] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417481] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417485] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417488] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417491] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417495] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417498] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417501] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417505] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417509] [ 790] 101 790 63477 626 1 0 0 rsyslogd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417512] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417515] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417519] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417522] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417525] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417528] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417531] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417534] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417537] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417540] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417544] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417547] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417550] [ 658] 1012 658 134393 70798 0 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417553] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417556] [13468] 0 13468 89584 2511 1 0 0 php5-fpm
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417559] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417562] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417565] [ 2686] 1012 2686 68618 5647 0 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417569] [ 2329] 108 2329 6818 73 1 0 0 cleanup
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417572] [ 2330] 108 2330 6795 71 0 0 0 trivial-rewrite
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417575] [ 2331] 108 2331 11107 159 0 0 0 smtp
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417578] [ 3728] 108 3728 6792 70 0 0 0 pickup
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417582] [ 3729] 108 3729 11107 141 1 0 0 smtp
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417585] [ 3856] 1012 3856 891340 828238 0 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417588] [ 4819] 1012 4819 68580 5604 1 0 0 php
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417591] Out of memory: Kill process 3856 (php) score 864 or sacrifice child
Jul 15 06:21:55 ip-10-61-174-46 kernel: [1396211.417601] Killed process 3856 (php) total-vm:3565360kB, anon-rss:3312868kB, file-rss:84kB
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372283] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372287] php cpuset=/ mems_allowed=0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372290] Pid: 5244, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372292] Call Trace:
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372299] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372301] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372303] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372306] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372309] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372313] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372315] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372318] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372321] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372323] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372327] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372329] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372331] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372334] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372336] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372340] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372342] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372344] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372347] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372349] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372351] Mem-Info:
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372352] Node 0 DMA per-cpu:
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372354] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372355] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372357] Node 0 DMA32 per-cpu:
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372358] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372359] CPU 1: hi: 186, btch: 31 usd: 27
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372362] active_anon:932189 inactive_anon:1393 isolated_anon:0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372363] active_file:0 inactive_file:0 isolated_file:0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372363] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372364] free:4038 slab_reclaimable:2955 slab_unreclaimable:2685
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372365] mapped:1365 shmem:1405 pagetables:3352 bounce:0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372366] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372373] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372376] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3728756kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:11820kB slab_unreclaimable:10740kB kernel_stack:848kB pagetables:13408kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:74 all_unreclaimable? yes
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372382] lowmem_reserve[]: 0 0 0 0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372385] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372391] Node 0 DMA32: 1442*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7816kB
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372398] 1423 total pagecache pages
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372399] 0 pages in swap cache
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372400] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372401] Free swap = 0kB
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.372402] Total swap = 0kB
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377495] 985072 pages RAM
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377498] 24998 pages reserved
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377499] 3808 pages shared
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377500] 953267 pages non-shared
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377502] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377511] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377514] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377517] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377520] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377523] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377526] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377528] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377532] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377535] [ 790] 101 790 63477 626 0 0 0 rsyslogd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377538] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377541] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377544] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377547] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377550] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377553] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377556] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377559] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377562] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377566] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377569] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377572] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377575] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377578] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377581] [13468] 0 13468 89584 2540 0 0 0 php5-fpm
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377584] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377587] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377590] [ 2686] 1012 2686 68618 5647 1 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377594] [ 3728] 108 3728 6792 76 0 0 0 pickup
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377597] [ 5244] 1012 5244 889997 826857 0 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377599] [ 8313] 0 8313 8460 79 1 0 0 cron
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377602] [ 8316] 0 8316 1099 26 1 0 0 sh
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377605] [ 8317] 0 8317 1074 26 0 0 0 run-parts
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377608] [ 8321] 0 8321 1099 35 0 0 0 apt
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377611] [ 8362] 0 8362 1469 24 0 0 0 sleep
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377615] [29567] 108 29567 6818 72 0 0 0 cleanup
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377618] [29568] 108 29568 6795 70 1 0 0 trivial-rewrite
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377621] [29569] 108 29569 11107 159 1 0 0 smtp
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377624] [29576] 108 29576 11076 150 1 0 0 smtp
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377627] [29591] 108 29591 11076 143 0 0 0 smtp
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377630] [29600] 108 29600 11107 158 0 0 0 smtp
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377633] [29605] 108 29605 11107 157 0 0 0 smtp
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377636] [29739] 108 29739 11108 160 0 0 0 smtp
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377639] [29740] 108 29740 6792 69 1 0 0 scache
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377642] [29780] 108 29780 6800 69 1 0 0 bounce
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377645] [29901] 1012 29901 68580 5604 0 0 0 php
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377648] Out of memory: Kill process 5244 (php) score 863 or sacrifice child
Jul 15 06:47:38 ip-10-61-174-46 kernel: [1397754.377659] Killed process 5244 (php) total-vm:3559988kB, anon-rss:3307428kB, file-rss:0kB
Jul 15 06:48:57 ip-10-61-174-46 kernel: [1397833.528163] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528167] php cpuset=/ mems_allowed=0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528170] Pid: 30258, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528172] Call Trace:
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528178] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528181] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528183] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528186] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528189] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528192] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528194] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528197] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528200] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528203] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528205] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528208] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528211] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528213] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528216] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528219] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528220] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528223] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528226] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528228] Mem-Info:
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528229] Node 0 DMA per-cpu:
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528231] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528232] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528233] Node 0 DMA32 per-cpu:
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528235] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528236] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528239] active_anon:932703 inactive_anon:1393 isolated_anon:0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528239] active_file:0 inactive_file:0 isolated_file:0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528240] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528240] free:4048 slab_reclaimable:2554 slab_unreclaimable:2643
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528241] mapped:1374 shmem:1405 pagetables:3277 bounce:0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528249] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528252] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730812kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5620kB slab_reclaimable:10216kB slab_unreclaimable:10572kB kernel_stack:824kB pagetables:13108kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528259] lowmem_reserve[]: 0 0 0 0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528261] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528268] Node 0 DMA32: 883*4kB 102*8kB 24*16kB 32*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528275] 1439 total pagecache pages
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528276] 0 pages in swap cache
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528277] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528278] Free swap = 0kB
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.528279] Total swap = 0kB
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533462] 985072 pages RAM
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533464] 24998 pages reserved
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533465] 3823 pages shared
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533466] 953285 pages non-shared
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533469] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533477] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533481] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533483] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533486] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533489] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533492] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533495] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533498] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533501] [ 790] 101 790 63477 626 0 0 0 rsyslogd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533504] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533507] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533511] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533514] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533517] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533520] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533523] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533526] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533529] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533533] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533536] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533539] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533541] [ 658] 1012 658 134393 70798 1 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533545] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533548] [13468] 0 13468 89584 2525 0 0 0 php5-fpm
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533551] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533554] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533557] [ 2686] 1012 2686 68618 5646 1 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533561] [ 3728] 108 3728 6792 76 0 0 0 pickup
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533564] [ 8313] 0 8313 8460 79 1 0 0 cron
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533567] [ 8316] 0 8316 1099 26 1 0 0 sh
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533570] [ 8317] 0 8317 1074 26 0 0 0 run-parts
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533573] [ 8321] 0 8321 1099 35 0 0 0 apt
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533576] [ 8362] 0 8362 1469 24 0 0 0 sleep
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533579] [29569] 108 29569 11107 159 0 0 0 smtp
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533582] [29576] 108 29576 11076 150 0 0 0 smtp
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533585] [29600] 108 29600 11107 158 0 0 0 smtp
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533588] [29605] 108 29605 11107 157 0 0 0 smtp
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533591] [29739] 108 29739 11108 160 0 0 0 smtp
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533593] [29740] 108 29740 6792 69 0 0 0 scache
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533596] [29780] 108 29780 6800 69 1 0 0 bounce
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533599] [30258] 1012 30258 890764 827677 0 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533602] [31138] 1012 31138 68580 5605 0 0 0 php
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533605] Out of memory: Kill process 30258 (php) score 863 or sacrifice child
Jul 15 06:48:58 ip-10-61-174-46 kernel: [1397833.533616] Killed process 30258 (php) total-vm:3563056kB, anon-rss:3310664kB, file-rss:44kB
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668144] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668149] php cpuset=/ mems_allowed=0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668153] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668154] Call Trace:
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668161] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668164] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668166] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668169] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668172] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668175] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668185] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668188] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668191] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668194] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668196] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668199] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668201] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668203] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668205] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668209] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668212] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668215] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668217] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668219] Mem-Info:
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668221] Node 0 DMA per-cpu:
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668222] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668225] Node 0 DMA32 per-cpu:
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668226] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668228] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668230] active_anon:932238 inactive_anon:1393 isolated_anon:0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668231] active_file:11 inactive_file:6 isolated_file:0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668232] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668232] free:4048 slab_reclaimable:2867 slab_unreclaimable:2692
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668233] mapped:1362 shmem:1405 pagetables:3380 bounce:0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668241] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668244] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728952kB inactive_anon:5572kB active_file:44kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:11468kB slab_unreclaimable:10768kB kernel_stack:816kB pagetables:13520kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2646 all_unreclaimable? yes
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668250] lowmem_reserve[]: 0 0 0 0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668259] Node 0 DMA32: 1470*4kB 9*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8000kB
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668266] 1407 total pagecache pages
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668267] 0 pages in swap cache
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668268] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668269] Free swap = 0kB
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.668270] Total swap = 0kB
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673569] 985072 pages RAM
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673572] 24998 pages reserved
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673573] 3775 pages shared
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673574] 953315 pages non-shared
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673576] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673585] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673588] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673591] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673594] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673598] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673601] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673604] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673608] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673612] [ 790] 101 790 63477 626 1 0 0 rsyslogd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673615] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673618] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673621] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673624] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673627] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673631] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673634] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673637] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673640] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673643] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673647] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673650] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673653] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673656] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673659] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673662] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673665] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673668] [ 2686] 1012 2686 68618 5647 0 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673671] [17863] 108 17863 6792 69 1 0 0 pickup
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673675] [ 5425] 1012 5425 879249 816299 1 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673678] [14040] 108 14040 6818 74 0 0 0 cleanup
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673681] [14041] 108 14041 6795 74 0 0 0 trivial-rewrite
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673684] [14042] 108 14042 11107 158 1 0 0 smtp
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673687] [14046] 108 14046 11107 159 1 0 0 smtp
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673690] [14079] 1012 14079 68580 5603 0 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673693] [14233] 1012 14233 1099 24 0 0 0 sh
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673696] [14234] 1012 14234 68825 5867 1 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673700] [14241] 1012 14241 1099 25 0 0 0 sh
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673703] [14242] 1012 14242 4724 58 0 0 0 git
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673705] [14243] 1012 14243 68499 5552 0 0 0 php
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673709] [14245] 1012 14245 1099 24 1 0 0 sh
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673711] Out of memory: Kill process 5425 (php) score 852 or sacrifice child
Jul 15 07:43:05 ip-10-61-174-46 kernel: [1401080.673722] Killed process 5425 (php) total-vm:3516996kB, anon-rss:3265196kB, file-rss:0kB
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116011] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116015] ntpd cpuset=/ mems_allowed=0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116018] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116020] Call Trace:
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116027] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116029] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116032] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116034] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116038] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116041] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116043] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116046] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116048] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116053] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116070] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116074] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116078] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116082] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116086] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116088] Mem-Info:
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116090] Node 0 DMA per-cpu:
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116092] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116094] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116096] Node 0 DMA32 per-cpu:
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116098] CPU 0: hi: 186, btch: 31 usd: 68
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116100] CPU 1: hi: 186, btch: 31 usd: 4
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116104] active_anon:932722 inactive_anon:1393 isolated_anon:0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116105] active_file:0 inactive_file:4 isolated_file:12
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116106] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116106] free:4047 slab_reclaimable:2602 slab_unreclaimable:2663
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116107] mapped:1357 shmem:1405 pagetables:3160 bounce:0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116109] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116118] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116122] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3730888kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10408kB slab_unreclaimable:10652kB kernel_stack:768kB pagetables:12640kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:216 all_unreclaimable? yes
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116130] lowmem_reserve[]: 0 0 0 0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116134] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116144] Node 0 DMA32: 939*4kB 132*8kB 54*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116154] 1446 total pagecache pages
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116156] 0 pages in swap cache
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116157] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116159] Free swap = 0kB
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.116160] Total swap = 0kB
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121438] 985072 pages RAM
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121440] 24998 pages reserved
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121442] 3783 pages shared
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121443] 953278 pages non-shared
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121444] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121454] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121457] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121459] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121462] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121465] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121468] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121471] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121474] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121477] [ 790] 101 790 63477 626 0 0 0 rsyslogd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121480] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121482] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121484] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121486] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121488] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121490] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121493] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121495] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121497] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121499] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121502] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121504] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121506] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121508] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121511] [13468] 0 13468 89584 2499 0 0 0 php5-fpm
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121513] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121515] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121517] [ 2686] 1012 2686 68618 5646 0 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121520] [17863] 108 17863 6792 75 0 0 0 pickup
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121522] [14040] 108 14040 6818 82 0 0 0 cleanup
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121524] [14041] 108 14041 6795 74 0 0 0 trivial-rewrite
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121526] [14042] 108 14042 11107 157 1 0 0 smtp
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121528] [14046] 108 14046 11107 158 1 0 0 smtp
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121531] [14388] 1012 14388 891510 828312 1 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121533] [14917] 1012 14917 68580 5604 1 0 0 php
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121535] Out of memory: Kill process 14388 (php) score 864 or sacrifice child
Jul 15 07:44:04 ip-10-61-174-46 kernel: [1401140.121544] Killed process 14388 (php) total-vm:3566040kB, anon-rss:3313248kB, file-rss:0kB
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900393] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900397] php cpuset=/ mems_allowed=0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900400] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900402] Call Trace:
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900409] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900411] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900413] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900416] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900419] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900423] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900425] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900428] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900431] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900434] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900436] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900438] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900441] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900445] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900448] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900452] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900455] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900458] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900460] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900463] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900466] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900467] Mem-Info:
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900468] Node 0 DMA per-cpu:
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900470] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900472] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900473] Node 0 DMA32 per-cpu:
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900474] CPU 0: hi: 186, btch: 31 usd: 34
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900476] CPU 1: hi: 186, btch: 31 usd: 62
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900478] active_anon:932856 inactive_anon:1393 isolated_anon:0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900479] active_file:0 inactive_file:10 isolated_file:0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900480] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900480] free:4034 slab_reclaimable:2547 slab_unreclaimable:2617
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900481] mapped:1381 shmem:1405 pagetables:3156 bounce:0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900483] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900489] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900492] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3731424kB inactive_anon:5572kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:10188kB slab_unreclaimable:10468kB kernel_stack:760kB pagetables:12624kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:304 all_unreclaimable? yes
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900499] lowmem_reserve[]: 0 0 0 0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900501] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900508] Node 0 DMA32: 886*4kB 261*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900515] 1428 total pagecache pages
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900516] 0 pages in swap cache
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900517] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900518] Free swap = 0kB
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.900519] Total swap = 0kB
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905706] 985072 pages RAM
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905708] 24998 pages reserved
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905710] 3787 pages shared
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905711] 953262 pages non-shared
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905713] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905721] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905725] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905728] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905731] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905734] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905738] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905741] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905745] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905749] [ 790] 101 790 63477 626 0 0 0 rsyslogd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905752] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905755] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905758] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905761] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905764] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905767] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905770] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905773] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905777] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905780] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905783] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905786] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905789] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905792] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905796] [13468] 0 13468 89584 2528 0 0 0 php5-fpm
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905799] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905802] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905805] [ 2686] 1012 2686 68618 5656 1 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905808] [17863] 108 17863 6792 75 1 0 0 pickup
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905812] [15141] 1012 15141 891631 828443 0 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905815] [23486] 108 23486 6818 74 0 0 0 cleanup
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905819] [23487] 108 23487 6795 71 0 0 0 trivial-rewrite
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905822] [23488] 108 23488 11107 158 0 0 0 smtp
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905825] [23501] 108 23501 11107 157 1 0 0 smtp
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905828] [24632] 1012 24632 68580 5598 1 0 0 php
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905831] Out of memory: Kill process 15141 (php) score 864 or sacrifice child
Jul 15 07:55:40 ip-10-61-174-46 kernel: [1401835.905841] Killed process 15141 (php) total-vm:3566524kB, anon-rss:3313772kB, file-rss:0kB
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766336] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766340] php cpuset=/ mems_allowed=0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766343] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766344] Call Trace:
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766351] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766354] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766356] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766359] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766364] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766367] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766370] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766372] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766375] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766378] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766381] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766383] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766385] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766388] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766390] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766394] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766397] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766400] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766404] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766407] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766410] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766413] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766414] Mem-Info:
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766415] Node 0 DMA per-cpu:
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766417] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766419] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766420] Node 0 DMA32 per-cpu:
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766421] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766423] CPU 1: hi: 186, btch: 31 usd: 31
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766425] active_anon:932864 inactive_anon:1393 isolated_anon:0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766426] active_file:11 inactive_file:10 isolated_file:0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766427] unevictable:0 dirty:17 writeback:0 unstable:0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766427] free:4018 slab_reclaimable:2604 slab_unreclaimable:2629
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766428] mapped:1353 shmem:1405 pagetables:3172 bounce:0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766430] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766436] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766439] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3731456kB inactive_anon:5572kB active_file:44kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:68kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10416kB slab_unreclaimable:10516kB kernel_stack:768kB pagetables:12688kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:175 all_unreclaimable? yes
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766446] lowmem_reserve[]: 0 0 0 0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766448] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766455] Node 0 DMA32: 980*4kB 167*8kB 27*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766462] 1417 total pagecache pages
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766463] 0 pages in swap cache
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766464] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766465] Free swap = 0kB
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.766466] Total swap = 0kB
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771650] 985072 pages RAM
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771652] 24998 pages reserved
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771653] 3784 pages shared
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771655] 953317 pages non-shared
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771657] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771665] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771669] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771672] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771675] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771678] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771682] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771685] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771688] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771692] [ 790] 101 790 63477 626 0 0 0 rsyslogd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771695] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771698] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771702] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771705] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771708] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771711] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771714] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771717] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771720] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771723] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771727] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771729] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771732] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771735] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771738] [13468] 0 13468 89584 2513 1 0 0 php5-fpm
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771741] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771744] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771748] [ 2686] 1012 2686 68618 5656 1 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771751] [17863] 108 17863 6792 75 0 0 0 pickup
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771755] [23486] 108 23486 6818 82 0 0 0 cleanup
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771758] [23487] 108 23487 6795 75 1 0 0 trivial-rewrite
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771761] [23488] 108 23488 11107 158 1 0 0 smtp
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771764] [23501] 108 23501 11107 156 1 0 0 smtp
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771767] [24632] 1012 24632 886918 823825 0 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771770] [24675] 108 24675 11107 159 1 0 0 smtp
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771774] [24676] 1012 24676 73006 10075 1 0 0 php
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771776] Out of memory: Kill process 24632 (php) score 859 or sacrifice child
Jul 15 07:56:15 ip-10-61-174-46 kernel: [1401870.771788] Killed process 24632 (php) total-vm:3547672kB, anon-rss:3295300kB, file-rss:0kB
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936144] php invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936148] php cpuset=/ mems_allowed=0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936150] Pid: 13526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936152] Call Trace:
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936160] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936162] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936164] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936167] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936170] [<ffffffff8111804a>] ? unlock_page+0x2a/0x40
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936174] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936176] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936180] [<ffffffff81661f78>] ? do_page_fault+0x288/0x550
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936182] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936184] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936187] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936190] [<ffffffff81177941>] ? fd_install+0x61/0x80
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936193] [<ffffffff81183c33>] ? do_pipe_flags+0xc3/0x120
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936196] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936199] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936202] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936203] Mem-Info:
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936205] Node 0 DMA per-cpu:
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936207] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936208] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936209] Node 0 DMA32 per-cpu:
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936211] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936212] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936215] active_anon:932997 inactive_anon:1393 isolated_anon:0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936215] active_file:28 inactive_file:1 isolated_file:0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936216] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936217] free:4199 slab_reclaimable:2311 slab_unreclaimable:2645
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936217] mapped:1349 shmem:1405 pagetables:3145 bounce:0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936219] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936226] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936228] Node 0 DMA32 free:8460kB min:7848kB low:9808kB high:11772kB active_anon:3731988kB inactive_anon:5572kB active_file:112kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5396kB shmem:5620kB slab_reclaimable:9244kB slab_unreclaimable:10580kB kernel_stack:760kB pagetables:12580kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1636 all_unreclaimable? yes
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936235] lowmem_reserve[]: 0 0 0 0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936237] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936244] Node 0 DMA32: 1479*4kB 66*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8492kB
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936251] 1439 total pagecache pages
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936252] 0 pages in swap cache
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936253] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936254] Free swap = 0kB
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.936255] Total swap = 0kB
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941531] 985072 pages RAM
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941533] 24998 pages reserved
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941534] 3774 pages shared
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941535] 953200 pages non-shared
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941537] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941546] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941549] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941553] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941556] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941559] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941562] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941565] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941569] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941573] [ 790] 101 790 63477 629 0 0 0 rsyslogd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941576] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941579] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941583] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941586] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941589] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941593] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941596] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941600] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941603] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941606] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941609] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941612] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941616] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941618] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941622] [13468] 0 13468 89584 2533 1 0 0 php5-fpm
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941625] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941627] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941631] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941634] [17863] 108 17863 6792 75 1 0 0 pickup
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941638] [ 1278] 1012 1278 891690 828463 1 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941641] [13216] 108 13216 6818 74 0 0 0 cleanup
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941644] [13217] 108 13217 6795 69 0 0 0 trivial-rewrite
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941647] [13218] 108 13218 11107 159 0 0 0 smtp
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941650] [13525] 1012 13525 1099 24 0 0 0 sh
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941653] [13526] 1012 13526 68825 5849 0 0 0 php
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941656] Out of memory: Kill process 1278 (php) score 864 or sacrifice child
Jul 15 08:22:52 ip-10-61-174-46 kernel: [1403467.941667] Killed process 1278 (php) total-vm:3566760kB, anon-rss:3313852kB, file-rss:0kB
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904152] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904156] php cpuset=/ mems_allowed=0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904159] Pid: 15682, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904161] Call Trace:
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904169] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904172] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904175] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904180] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904186] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904189] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904193] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904195] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904198] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904201] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904204] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904206] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904208] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904211] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904213] [<ffffffff811425b1>] ? find_vma+0x1/0x70
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904216] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904219] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904224] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904226] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904229] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904232] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904233] Mem-Info:
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904235] Node 0 DMA per-cpu:
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904237] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904238] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904239] Node 0 DMA32 per-cpu:
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904241] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904242] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904245] active_anon:932944 inactive_anon:1393 isolated_anon:0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904245] active_file:25 inactive_file:32 isolated_file:0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904246] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904247] free:4047 slab_reclaimable:2571 slab_unreclaimable:2602
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904247] mapped:1357 shmem:1405 pagetables:3121 bounce:0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904249] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904255] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904258] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731776kB inactive_anon:5572kB active_file:100kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10284kB slab_unreclaimable:10408kB kernel_stack:744kB pagetables:12484kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:22664 all_unreclaimable? yes
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904265] lowmem_reserve[]: 0 0 0 0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904267] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904274] Node 0 DMA32: 936*4kB 166*8kB 42*16kB 4*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904281] 1443 total pagecache pages
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904282] 0 pages in swap cache
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904283] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904284] Free swap = 0kB
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.904285] Total swap = 0kB
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909558] 985072 pages RAM
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909561] 24998 pages reserved
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909562] 3774 pages shared
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909563] 953329 pages non-shared
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909565] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909573] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909577] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909580] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909583] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909586] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909589] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909593] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909596] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909600] [ 790] 101 790 63477 629 0 0 0 rsyslogd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909603] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909606] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909609] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909612] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909616] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909619] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909622] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909625] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909628] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909631] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909634] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909638] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909641] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909644] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909647] [13468] 0 13468 89584 2507 0 0 0 php5-fpm
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909650] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909653] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909656] [ 2686] 1012 2686 68618 5660 0 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909660] [13618] 1012 13618 891847 828753 1 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909663] [15399] 108 15399 6818 73 0 0 0 cleanup
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909666] [15400] 108 15400 6795 71 0 0 0 trivial-rewrite
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909669] [15401] 108 15401 11107 158 1 0 0 smtp
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909672] [15682] 1012 15682 68580 5604 0 0 0 php
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909675] Out of memory: Kill process 13618 (php) score 865 or sacrifice child
Jul 15 08:25:55 ip-10-61-174-46 kernel: [1403650.909686] Killed process 13618 (php) total-vm:3567388kB, anon-rss:3315012kB, file-rss:0kB
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756114] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756118] php cpuset=/ mems_allowed=0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756121] Pid: 15940, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756123] Call Trace:
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756130] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756132] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756135] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756138] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756141] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756144] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756147] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756150] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756153] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756156] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756159] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756161] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756163] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756166] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756169] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756172] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756174] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756177] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756180] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756181] Mem-Info:
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756183] Node 0 DMA per-cpu:
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756185] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756186] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756187] Node 0 DMA32 per-cpu:
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756189] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756190] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756193] active_anon:933016 inactive_anon:1393 isolated_anon:0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756193] active_file:8 inactive_file:16 isolated_file:0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756194] unevictable:0 dirty:9 writeback:0 unstable:0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756195] free:4048 slab_reclaimable:2480 slab_unreclaimable:2627
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756195] mapped:1369 shmem:1405 pagetables:3154 bounce:0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756197] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756204] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756207] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732064kB inactive_anon:5572kB active_file:32kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:36kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:9920kB slab_unreclaimable:10508kB kernel_stack:760kB pagetables:12616kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8199 all_unreclaimable? yes
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756213] lowmem_reserve[]: 0 0 0 0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756216] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756222] Node 0 DMA32: 901*4kB 207*8kB 30*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756229] 1463 total pagecache pages
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756231] 0 pages in swap cache
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756232] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756233] Free swap = 0kB
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.756234] Total swap = 0kB
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761587] 985072 pages RAM
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761590] 24998 pages reserved
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761591] 3801 pages shared
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761592] 953325 pages non-shared
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761594] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761602] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761606] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761609] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761612] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761615] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761618] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761622] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761625] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761629] [ 790] 101 790 63477 629 0 0 0 rsyslogd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761632] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761635] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761638] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761641] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761645] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761648] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761651] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761654] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761657] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761660] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761664] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761667] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761670] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761673] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761676] [13468] 0 13468 89584 2492 1 0 0 php5-fpm
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761679] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761682] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761685] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761689] [15399] 108 15399 6818 81 0 0 0 cleanup
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761692] [15400] 108 15400 6795 75 0 0 0 trivial-rewrite
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761695] [15401] 108 15401 11107 158 0 0 0 smtp
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761698] [15940] 1012 15940 889708 826718 0 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761701] [16174] 108 16174 6792 70 1 0 0 pickup
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761705] [16185] 108 16185 11107 158 0 0 0 smtp
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761708] [17499] 1012 17499 70448 7489 1 0 0 php
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761711] Out of memory: Kill process 15940 (php) score 862 or sacrifice child
Jul 15 08:28:19 ip-10-61-174-46 kernel: [1403794.761722] Killed process 15940 (php) total-vm:3558832kB, anon-rss:3306768kB, file-rss:104kB
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116013] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116017] ntpd cpuset=/ mems_allowed=0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116020] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116022] Call Trace:
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116029] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116031] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116033] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116036] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116039] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116043] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116045] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116048] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116052] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116067] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116070] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116074] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116078] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116082] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116086] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116088] Mem-Info:
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116090] Node 0 DMA per-cpu:
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116092] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116094] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116096] Node 0 DMA32 per-cpu:
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116098] CPU 0: hi: 186, btch: 31 usd: 106
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116100] CPU 1: hi: 186, btch: 31 usd: 43
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116104] active_anon:932770 inactive_anon:1393 isolated_anon:0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116105] active_file:9 inactive_file:18 isolated_file:20
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116106] unevictable:0 dirty:2 writeback:0 unstable:0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116107] free:4044 slab_reclaimable:2440 slab_unreclaimable:2585
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116108] mapped:1358 shmem:1405 pagetables:3253 bounce:0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116110] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116118] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116122] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731080kB inactive_anon:5572kB active_file:36kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:8kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9760kB slab_unreclaimable:10340kB kernel_stack:792kB pagetables:13012kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:209 all_unreclaimable? yes
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116131] lowmem_reserve[]: 0 0 0 0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116135] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116145] Node 0 DMA32: 687*4kB 212*8kB 66*16kB 8*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116155] 1478 total pagecache pages
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116156] 0 pages in swap cache
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116158] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116160] Free swap = 0kB
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.116161] Total swap = 0kB
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121457] 985072 pages RAM
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121459] 24998 pages reserved
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121460] 3788 pages shared
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121461] 953189 pages non-shared
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121462] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121470] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121472] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121475] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121477] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121479] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121481] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121484] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121487] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121490] [ 790] 101 790 63477 629 0 0 0 rsyslogd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121492] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121494] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121497] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121499] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121501] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121503] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121505] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121508] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121510] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121512] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121514] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121516] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121519] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121521] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121523] [13468] 0 13468 89584 2526 1 0 0 php5-fpm
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121525] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121527] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121530] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121533] [16174] 108 16174 6792 76 1 0 0 pickup
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121535] [17806] 1012 17806 888533 825457 0 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121537] [20624] 108 20624 6818 74 0 0 0 cleanup
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121539] [20625] 108 20625 6795 70 1 0 0 trivial-rewrite
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121542] [20626] 108 20626 11107 158 0 0 0 smtp
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121544] [20872] 1012 20872 1099 25 1 0 0 sh
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121546] [20873] 1012 20873 68825 5870 1 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121548] [20879] 1012 20879 1099 26 1 0 0 sh
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121550] [20880] 1012 20880 4676 57 1 0 0 git
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121552] [20881] 1012 20881 65117 2699 0 0 0 php
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121554] Out of memory: Kill process 17806 (php) score 861 or sacrifice child
Jul 15 08:32:48 ip-10-61-174-46 kernel: [1404064.121564] Killed process 17806 (php) total-vm:3554132kB, anon-rss:3301828kB, file-rss:0kB
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592343] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592347] php cpuset=/ mems_allowed=0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592350] Pid: 32189, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592352] Call Trace:
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592359] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592361] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592363] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592366] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592370] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592373] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592375] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592378] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592381] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592384] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592386] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592389] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592393] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592395] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592397] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592400] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592401] Mem-Info:
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592403] Node 0 DMA per-cpu:
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592404] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592406] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592407] Node 0 DMA32 per-cpu:
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592409] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592410] CPU 1: hi: 186, btch: 31 usd: 50
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592413] active_anon:932885 inactive_anon:1393 isolated_anon:0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592413] active_file:28 inactive_file:6 isolated_file:0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592414] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592415] free:4034 slab_reclaimable:2642 slab_unreclaimable:2587
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592415] mapped:1359 shmem:1405 pagetables:3113 bounce:0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592417] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592423] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592426] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3731540kB inactive_anon:5572kB active_file:112kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:10568kB slab_unreclaimable:10348kB kernel_stack:728kB pagetables:12452kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:280 all_unreclaimable? yes
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592433] lowmem_reserve[]: 0 0 0 0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592435] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592442] Node 0 DMA32: 803*4kB 278*8kB 17*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592449] 1426 total pagecache pages
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592450] 0 pages in swap cache
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592451] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592452] Free swap = 0kB
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.592453] Total swap = 0kB
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597797] 985072 pages RAM
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597799] 24998 pages reserved
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597800] 3785 pages shared
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597801] 953295 pages non-shared
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597803] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597812] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597816] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597819] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597822] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597825] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597828] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597832] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597835] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597839] [ 790] 101 790 63477 629 0 0 0 rsyslogd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597842] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597845] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597849] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597852] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597855] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597858] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597861] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597864] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597867] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597871] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597874] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597877] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597880] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597883] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597886] [13468] 0 13468 89584 2527 1 0 0 php5-fpm
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597889] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597892] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597895] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597899] [16174] 108 16174 6792 76 1 0 0 pickup
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597902] [20999] 1012 20999 891667 828552 0 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597905] [32036] 1012 32036 68580 5604 1 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597908] [32188] 1012 32188 1099 25 1 0 0 sh
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597911] [32189] 1012 32189 31456 379 0 0 0 php
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597914] Out of memory: Kill process 20999 (php) score 864 or sacrifice child
Jul 15 08:48:23 ip-10-61-174-46 kernel: [1404998.597925] Killed process 20999 (php) total-vm:3566668kB, anon-rss:3314208kB, file-rss:0kB
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080111] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080115] php cpuset=/ mems_allowed=0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080117] Pid: 32264, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080119] Call Trace:
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080126] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080128] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080130] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080133] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080137] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080140] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080142] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080146] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080148] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080150] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080154] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080156] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080158] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080161] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080164] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080166] Mem-Info:
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080167] Node 0 DMA per-cpu:
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080169] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080170] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080171] Node 0 DMA32 per-cpu:
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080173] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080174] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080176] active_anon:933123 inactive_anon:1393 isolated_anon:0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080177] active_file:0 inactive_file:0 isolated_file:0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080178] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080178] free:4048 slab_reclaimable:2378 slab_unreclaimable:2583
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080179] mapped:1364 shmem:1405 pagetables:3171 bounce:0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080181] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080187] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080190] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732492kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:9512kB slab_unreclaimable:10332kB kernel_stack:760kB pagetables:12684kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080197] lowmem_reserve[]: 0 0 0 0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080199] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080206] Node 0 DMA32: 1189*4kB 132*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7924kB
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080212] 1458 total pagecache pages
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080214] 0 pages in swap cache
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080215] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080216] Free swap = 0kB
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.080217] Total swap = 0kB
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085737] 985072 pages RAM
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085739] 24998 pages reserved
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085740] 3802 pages shared
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085741] 953317 pages non-shared
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085743] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085752] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085755] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085758] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085761] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085765] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085768] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085771] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085774] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085779] [ 790] 101 790 63477 629 1 0 0 rsyslogd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085782] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085785] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085788] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085791] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085794] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085797] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085800] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085804] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085807] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085810] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085813] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085816] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085819] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085822] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085825] [13468] 0 13468 89584 2512 1 0 0 php5-fpm
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085828] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085831] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085834] [ 2686] 1012 2686 68618 5661 0 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085837] [16174] 108 16174 6792 76 1 0 0 pickup
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085841] [32264] 1012 32264 886530 823287 0 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085844] [29752] 1012 29752 68580 5606 1 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085847] [30013] 1012 30013 1099 25 0 0 0 sh
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085850] [30014] 1012 30014 68825 5858 0 0 0 php
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085853] Out of memory: Kill process 32264 (php) score 859 or sacrifice child
Jul 15 09:30:09 ip-10-61-174-46 kernel: [1407505.085864] Killed process 32264 (php) total-vm:3546120kB, anon-rss:3293136kB, file-rss:12kB
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704120] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704124] php cpuset=/ mems_allowed=0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704126] Pid: 31539, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704128] Call Trace:
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704135] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704137] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704139] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704142] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704146] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704149] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704151] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704154] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704157] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704159] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704163] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704165] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704167] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704170] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704173] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704174] Mem-Info:
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704176] Node 0 DMA per-cpu:
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704178] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704179] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704180] Node 0 DMA32 per-cpu:
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704182] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704183] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704185] active_anon:932991 inactive_anon:1393 isolated_anon:0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704186] active_file:28 inactive_file:30 isolated_file:0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704187] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704188] free:4048 slab_reclaimable:2594 slab_unreclaimable:2605
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704188] mapped:1386 shmem:1405 pagetables:3072 bounce:0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704190] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704196] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704199] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731964kB inactive_anon:5572kB active_file:112kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:10376kB slab_unreclaimable:10420kB kernel_stack:744kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6986 all_unreclaimable? yes
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704206] lowmem_reserve[]: 0 0 0 0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704208] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704215] Node 0 DMA32: 1058*4kB 182*8kB 5*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7912kB
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704222] 1479 total pagecache pages
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704223] 0 pages in swap cache
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704224] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704225] Free swap = 0kB
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.704226] Total swap = 0kB
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709427] 985072 pages RAM
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709429] 24998 pages reserved
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709430] 3808 pages shared
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709431] 953315 pages non-shared
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709433] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709442] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709445] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709448] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709451] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709455] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709458] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709461] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709464] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709469] [ 790] 101 790 63477 629 1 0 0 rsyslogd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709472] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709475] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709478] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709481] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709484] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709487] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709490] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709493] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709497] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709500] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709503] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709506] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709509] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709512] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709516] [13468] 0 13468 89584 2497 1 0 0 php5-fpm
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709519] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709521] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709525] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709528] [16174] 108 16174 6792 76 1 0 0 pickup
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709532] [31539] 1012 31539 892179 829048 0 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709535] [32253] 1012 32253 68580 5604 1 0 0 php
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709538] Out of memory: Kill process 31539 (php) score 865 or sacrifice child
Jul 15 09:33:35 ip-10-61-174-46 kernel: [1407710.709549] Killed process 31539 (php) total-vm:3568716kB, anon-rss:3316068kB, file-rss:124kB
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348596] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348600] php cpuset=/ mems_allowed=0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348603] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348604] Call Trace:
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348612] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348614] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348616] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348619] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348624] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348626] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348630] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348633] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348636] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348639] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348643] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348646] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348649] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348651] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348654] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348656] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348659] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348660] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348663] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348665] Mem-Info:
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348666] Node 0 DMA per-cpu:
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348668] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348669] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348671] Node 0 DMA32 per-cpu:
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348672] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348673] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348676] active_anon:932989 inactive_anon:1393 isolated_anon:0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348677] active_file:6 inactive_file:8 isolated_file:0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348677] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348678] free:4047 slab_reclaimable:2660 slab_unreclaimable:2639
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348679] mapped:1367 shmem:1405 pagetables:3055 bounce:0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348680] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348687] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348690] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731956kB inactive_anon:5572kB active_file:24kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5620kB slab_reclaimable:10640kB slab_unreclaimable:10556kB kernel_stack:728kB pagetables:12220kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:90 all_unreclaimable? yes
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348696] lowmem_reserve[]: 0 0 0 0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348699] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348705] Node 0 DMA32: 867*4kB 252*8kB 18*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348712] 1444 total pagecache pages
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348713] 0 pages in swap cache
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348715] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348716] Free swap = 0kB
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.348716] Total swap = 0kB
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353960] 985072 pages RAM
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353963] 24998 pages reserved
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353964] 3779 pages shared
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353966] 953330 pages non-shared
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353967] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353976] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353979] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353982] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353985] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353989] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353992] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353995] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.353998] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354003] [ 790] 101 790 63477 629 0 0 0 rsyslogd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354006] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354009] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354012] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354015] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354018] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354021] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354024] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354027] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354031] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354034] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354037] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354040] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354043] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354046] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354049] [13468] 0 13468 89584 2512 0 0 0 php5-fpm
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354052] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354055] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354059] [ 2686] 1012 2686 68618 5661 0 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354062] [16174] 108 16174 6792 76 1 0 0 pickup
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354066] [32442] 1012 32442 892209 829031 0 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354069] [ 6550] 1012 6550 68580 5603 0 0 0 php
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354071] Out of memory: Kill process 32442 (php) score 865 or sacrifice child
Jul 15 09:44:05 ip-10-61-174-46 kernel: [1408341.354082] Killed process 32442 (php) total-vm:3568836kB, anon-rss:3316124kB, file-rss:0kB
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204511] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204515] php cpuset=/ mems_allowed=0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204519] Pid: 7885, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204521] Call Trace:
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204529] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204533] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204535] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204539] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204544] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204550] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204553] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204557] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204562] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204565] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204569] [<ffffffff8100630a>] ? pte_mfn_to_pfn+0x3a/0x110
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204572] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204575] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204578] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204581] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204590] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204594] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204599] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204603] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204606] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204608] Mem-Info:
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204610] Node 0 DMA per-cpu:
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204612] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204614] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204616] Node 0 DMA32 per-cpu:
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204618] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204620] CPU 1: hi: 186, btch: 31 usd: 30
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204624] active_anon:932943 inactive_anon:1393 isolated_anon:0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204625] active_file:18 inactive_file:41 isolated_file:1
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204626] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204626] free:4033 slab_reclaimable:2308 slab_unreclaimable:2628
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204627] mapped:1328 shmem:1405 pagetables:3300 bounce:0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204630] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204638] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204642] Node 0 DMA32 free:7796kB min:7848kB low:9808kB high:11772kB active_anon:3731772kB inactive_anon:5572kB active_file:72kB inactive_file:164kB unevictable:0kB isolated(anon):0kB isolated(file):4kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5312kB shmem:5620kB slab_reclaimable:9232kB slab_unreclaimable:10512kB kernel_stack:776kB pagetables:13200kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204651] lowmem_reserve[]: 0 0 0 0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204654] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204664] Node 0 DMA32: 815*4kB 229*8kB 33*16kB 2*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204673] 1503 total pagecache pages
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204675] 0 pages in swap cache
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204676] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204678] Free swap = 0kB
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.204679] Total swap = 0kB
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209953] 985072 pages RAM
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209955] 24998 pages reserved
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209957] 3795 pages shared
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209958] 953322 pages non-shared
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209960] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209969] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209972] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209975] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209978] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209982] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209985] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209988] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209992] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209996] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.209999] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210002] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210005] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210008] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210012] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210015] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210018] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210021] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210024] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210027] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210030] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210033] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210036] [ 658] 1012 658 134393 70798 1 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210039] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210042] [13468] 0 13468 89584 2533 0 0 0 php5-fpm
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210045] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210048] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210052] [ 2686] 1012 2686 68618 5660 0 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210056] [ 6708] 1012 6708 880466 817334 1 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210059] [ 7822] 1012 7822 68580 5604 0 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210062] [ 7884] 1012 7884 1099 24 0 0 0 sh
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210065] [ 7885] 1012 7885 68825 5867 0 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210068] [ 7891] 1012 7891 1099 27 1 0 0 sh
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210071] [ 7892] 1012 7892 4655 57 1 0 0 git
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210074] [ 7893] 1012 7893 68499 5552 1 0 0 php
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210077] [ 7895] 1012 7895 1099 25 1 0 0 sh
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210080] [ 7896] 1012 7896 10901 165 1 0 0 ssh
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210082] Out of memory: Kill process 6708 (php) score 853 or sacrifice child
Jul 15 09:46:07 ip-10-61-174-46 kernel: [1408463.210093] Killed process 6708 (php) total-vm:3521864kB, anon-rss:3269336kB, file-rss:0kB
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271088] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271093] php cpuset=/ mems_allowed=0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271096] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271098] Call Trace:
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271105] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271107] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271110] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271112] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271116] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271119] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271121] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271125] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271128] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271130] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271133] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271136] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271138] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271141] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271144] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271147] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271150] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271152] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271155] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271157] Mem-Info:
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271158] Node 0 DMA per-cpu:
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271160] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271162] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271163] Node 0 DMA32 per-cpu:
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271164] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271166] CPU 1: hi: 186, btch: 31 usd: 65
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271168] active_anon:932737 inactive_anon:1393 isolated_anon:0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271169] active_file:7 inactive_file:10 isolated_file:0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271170] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271171] free:4045 slab_reclaimable:2771 slab_unreclaimable:2607
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271171] mapped:1390 shmem:1405 pagetables:3081 bounce:0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271173] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271179] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271182] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730948kB inactive_anon:5572kB active_file:28kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5560kB shmem:5620kB slab_reclaimable:11084kB slab_unreclaimable:10428kB kernel_stack:728kB pagetables:12324kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:115 all_unreclaimable? yes
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271189] lowmem_reserve[]: 0 0 0 0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271191] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271198] Node 0 DMA32: 737*4kB 170*8kB 39*16kB 17*32kB 5*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7844kB
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271205] 1442 total pagecache pages
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271206] 0 pages in swap cache
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271207] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271208] Free swap = 0kB
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.271209] Total swap = 0kB
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276317] 985072 pages RAM
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276320] 24998 pages reserved
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276321] 3802 pages shared
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276322] 953243 pages non-shared
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276324] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276333] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276336] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276339] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276342] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276346] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276349] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276352] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276355] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276359] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276362] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276365] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276369] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276372] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276375] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276378] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276381] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276384] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276387] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276391] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276394] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276397] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276400] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276403] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276406] [13468] 0 13468 89584 2518 0 0 0 php5-fpm
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276409] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276412] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276415] [ 2686] 1012 2686 68618 5660 0 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276419] [ 7987] 1012 7987 891955 828798 0 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276422] [ 8098] 108 8098 6792 70 0 0 0 pickup
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276425] [12061] 1012 12061 68580 5605 0 0 0 php
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276428] Out of memory: Kill process 7987 (php) score 865 or sacrifice child
Jul 15 09:52:28 ip-10-61-174-46 kernel: [1408844.276439] Killed process 7987 (php) total-vm:3567820kB, anon-rss:3315192kB, file-rss:0kB
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784127] php cpuset=/ mems_allowed=0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784130] Pid: 13559, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784132] Call Trace:
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784140] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784150] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784154] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784156] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784159] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784161] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784165] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784167] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784169] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784172] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784175] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784178] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784180] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784182] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784185] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784187] Mem-Info:
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784188] Node 0 DMA per-cpu:
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784190] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784191] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784192] Node 0 DMA32 per-cpu:
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784194] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784195] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784198] active_anon:932774 inactive_anon:1393 isolated_anon:0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784199] active_file:36 inactive_file:40 isolated_file:0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784199] unevictable:0 dirty:18 writeback:0 unstable:0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784200] free:4048 slab_reclaimable:2568 slab_unreclaimable:2619
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784201] mapped:1404 shmem:1405 pagetables:3242 bounce:0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784202] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784209] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784212] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731096kB inactive_anon:5572kB active_file:144kB inactive_file:160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:72kB writeback:0kB mapped:5616kB shmem:5620kB slab_reclaimable:10272kB slab_unreclaimable:10476kB kernel_stack:760kB pagetables:12968kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:20044 all_unreclaimable? yes
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784218] lowmem_reserve[]: 0 0 0 0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784221] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784227] Node 0 DMA32: 893*4kB 145*8kB 26*16kB 22*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8028kB
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784234] 1471 total pagecache pages
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784235] 0 pages in swap cache
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784237] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784238] Free swap = 0kB
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.784239] Total swap = 0kB
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789415] 985072 pages RAM
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789417] 24998 pages reserved
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789418] 3822 pages shared
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789419] 953276 pages non-shared
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789421] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789430] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789434] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789437] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789440] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789443] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789446] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789449] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789453] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789457] [ 790] 101 790 63477 630 1 0 0 rsyslogd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789460] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789463] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789466] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789469] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789473] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789476] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789479] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789482] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789485] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789488] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789491] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789494] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789497] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789500] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789504] [13468] 0 13468 89584 2523 0 0 0 php5-fpm
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789507] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789510] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789513] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789517] [ 8098] 108 8098 6792 70 0 0 0 pickup
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789520] [12322] 1012 12322 883667 820550 1 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789523] [13463] 1012 13463 68580 5604 1 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789526] [13558] 1012 13558 1099 24 0 0 0 sh
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789529] [13559] 1012 13559 66679 4300 0 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789532] [13560] 1012 13560 1099 25 0 0 0 sh
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789535] [13561] 1012 13561 66419 3969 1 0 0 php
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789538] Out of memory: Kill process 12322 (php) score 856 or sacrifice child
Jul 15 09:54:32 ip-10-61-174-46 kernel: [1408967.789549] Killed process 12322 (php) total-vm:3534668kB, anon-rss:3282200kB, file-rss:0kB
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026285] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026289] php cpuset=/ mems_allowed=0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026292] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026294] Call Trace:
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026301] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026303] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026305] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026308] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026311] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026315] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026317] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026320] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026323] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026325] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026329] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026331] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026334] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026337] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026340] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026343] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026345] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026348] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026350] Mem-Info:
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026351] Node 0 DMA per-cpu:
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026353] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026354] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026355] Node 0 DMA32 per-cpu:
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026357] CPU 0: hi: 186, btch: 31 usd: 4
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026358] CPU 1: hi: 186, btch: 31 usd: 61
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026361] active_anon:933352 inactive_anon:1393 isolated_anon:0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026362] active_file:4 inactive_file:0 isolated_file:0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026363] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026363] free:4047 slab_reclaimable:2328 slab_unreclaimable:2577
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026364] mapped:1373 shmem:1405 pagetables:3069 bounce:0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026366] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026372] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026375] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3733408kB inactive_anon:5572kB active_file:16kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5492kB shmem:5620kB slab_reclaimable:9312kB slab_unreclaimable:10308kB kernel_stack:728kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:125 all_unreclaimable? yes
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026381] lowmem_reserve[]: 0 0 0 0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026384] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026391] Node 0 DMA32: 709*4kB 155*8kB 34*16kB 31*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026397] 1426 total pagecache pages
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026398] 0 pages in swap cache
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026400] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026401] Free swap = 0kB
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.026402] Total swap = 0kB
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031646] 985072 pages RAM
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031648] 24998 pages reserved
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031649] 3772 pages shared
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031650] 953301 pages non-shared
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031652] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031661] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031665] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031668] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031671] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031674] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031677] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031681] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031684] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031688] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031691] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031695] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031698] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031701] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031704] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031707] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031710] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031713] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031716] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031720] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031723] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031726] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031729] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031732] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031735] [13468] 0 13468 89584 2508 0 0 0 php5-fpm
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031738] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031741] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031744] [ 2686] 1012 2686 68618 5660 0 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031748] [ 8098] 108 8098 6792 76 0 0 0 pickup
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031751] [13620] 1012 13620 892499 829403 0 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031754] [14209] 1012 14209 68580 5605 0 0 0 php
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031757] Out of memory: Kill process 13620 (php) score 865 or sacrifice child
Jul 15 09:55:38 ip-10-61-174-46 kernel: [1409034.031768] Killed process 13620 (php) total-vm:3569996kB, anon-rss:3317612kB, file-rss:0kB
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562647] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562654] php cpuset=/ mems_allowed=0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562657] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562660] Call Trace:
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562668] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562671] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562675] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562679] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562682] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562687] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562690] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562694] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562697] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562700] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562704] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562708] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562711] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562713] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562717] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562720] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562723] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562725] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562728] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562731] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562733] Mem-Info:
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562735] Node 0 DMA per-cpu:
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562736] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562738] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562739] Node 0 DMA32 per-cpu:
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562741] CPU 0: hi: 186, btch: 31 usd: 79
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562742] CPU 1: hi: 186, btch: 31 usd: 40
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562745] active_anon:932720 inactive_anon:1393 isolated_anon:0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562745] active_file:0 inactive_file:2 isolated_file:31
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562746] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562747] free:4045 slab_reclaimable:2497 slab_unreclaimable:2632
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562747] mapped:1365 shmem:1405 pagetables:3256 bounce:0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562749] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562755] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562758] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730880kB inactive_anon:5572kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):124kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:9988kB slab_unreclaimable:10528kB kernel_stack:744kB pagetables:13024kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:225 all_unreclaimable? yes
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562765] lowmem_reserve[]: 0 0 0 0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562767] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562774] Node 0 DMA32: 951*4kB 225*8kB 10*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7844kB
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562781] 1453 total pagecache pages
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562782] 0 pages in swap cache
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562783] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562784] Free swap = 0kB
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.562785] Total swap = 0kB
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568076] 985072 pages RAM
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568078] 24998 pages reserved
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568079] 3781 pages shared
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568080] 953235 pages non-shared
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568082] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568091] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568094] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568097] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568100] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568104] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568107] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568110] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568114] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568117] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568121] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568124] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568127] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568130] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568133] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568136] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568139] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568142] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568146] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568149] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568152] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568155] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568158] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568161] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568164] [13468] 0 13468 89584 2532 0 0 0 php5-fpm
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568167] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568170] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568173] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568177] [ 8098] 108 8098 6792 76 1 0 0 pickup
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568180] [14344] 1012 14344 885550 822379 0 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568184] [28739] 1012 28739 68580 5604 1 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568187] [29002] 1012 29002 1099 25 1 0 0 sh
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568190] [29003] 1012 29003 68825 5870 1 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568193] [29009] 1012 29009 1099 26 0 0 0 sh
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568196] [29010] 1012 29010 4736 65 0 0 0 git
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568199] [29011] 1012 29011 31456 384 0 0 0 php
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568202] Out of memory: Kill process 14344 (php) score 858 or sacrifice child
Jul 15 10:19:05 ip-10-61-174-46 kernel: [1410440.568213] Killed process 14344 (php) total-vm:3542200kB, anon-rss:3289516kB, file-rss:0kB
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684731] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684736] php cpuset=/ mems_allowed=0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684739] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684741] Call Trace:
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684749] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684752] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684755] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684759] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684765] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684768] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684773] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684776] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684780] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684784] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684787] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684792] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684795] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684799] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684803] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684807] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684810] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684813] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684817] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684819] Mem-Info:
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684821] Node 0 DMA per-cpu:
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684823] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684825] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684826] Node 0 DMA32 per-cpu:
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684829] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684831] CPU 1: hi: 186, btch: 31 usd: 95
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684834] active_anon:933063 inactive_anon:1393 isolated_anon:0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684835] active_file:0 inactive_file:0 isolated_file:0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684836] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684837] free:4028 slab_reclaimable:2445 slab_unreclaimable:2644
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684838] mapped:1356 shmem:1405 pagetables:3065 bounce:0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684840] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684848] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684852] Node 0 DMA32 free:7776kB min:7848kB low:9808kB high:11772kB active_anon:3732252kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9780kB slab_unreclaimable:10576kB kernel_stack:728kB pagetables:12260kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:358 all_unreclaimable? yes
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684861] lowmem_reserve[]: 0 0 0 0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684864] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684872] Node 0 DMA32: 902*4kB 199*8kB 25*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7776kB
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684878] 1406 total pagecache pages
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684880] 0 pages in swap cache
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684881] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684882] Free swap = 0kB
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.684883] Total swap = 0kB
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690259] 985072 pages RAM
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690262] 24998 pages reserved
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690263] 3775 pages shared
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690264] 953287 pages non-shared
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690266] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690274] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690278] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690281] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690284] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690288] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690291] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690294] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690297] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690301] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690305] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690308] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690311] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690314] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690317] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690320] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690324] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690327] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690330] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690333] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690336] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690339] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690342] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690345] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690348] [13468] 0 13468 89584 2517 0 0 0 php5-fpm
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690351] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690354] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690358] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690361] [ 8098] 108 8098 6792 76 1 0 0 pickup
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690365] [29034] 1012 29034 892243 829130 1 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690368] [30598] 1012 30598 68580 5605 0 0 0 php
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690371] Out of memory: Kill process 29034 (php) score 865 or sacrifice child
Jul 15 10:21:55 ip-10-61-174-46 kernel: [1410610.690382] Killed process 29034 (php) total-vm:3568972kB, anon-rss:3316520kB, file-rss:0kB
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056231] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056235] php cpuset=/ mems_allowed=0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056238] Pid: 848, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056240] Call Trace:
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056247] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056249] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056251] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056254] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056257] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056261] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056263] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056266] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056269] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056272] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056275] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056277] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056280] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056283] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056285] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056288] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056292] [<ffffffff8117e76a>] ? sys_newfstat+0x2a/0x40
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056295] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056297] Mem-Info:
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056298] Node 0 DMA per-cpu:
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056300] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056301] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056303] Node 0 DMA32 per-cpu:
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056304] CPU 0: hi: 186, btch: 31 usd: 33
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056306] CPU 1: hi: 186, btch: 31 usd: 62
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056308] active_anon:933063 inactive_anon:1393 isolated_anon:0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056309] active_file:37 inactive_file:0 isolated_file:0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056310] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056310] free:4028 slab_reclaimable:2372 slab_unreclaimable:2599
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056311] mapped:1342 shmem:1405 pagetables:3166 bounce:0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056312] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056319] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056322] Node 0 DMA32 free:7776kB min:7848kB low:9808kB high:11772kB active_anon:3732252kB inactive_anon:5572kB active_file:148kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5368kB shmem:5620kB slab_reclaimable:9488kB slab_unreclaimable:10396kB kernel_stack:752kB pagetables:12664kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:164 all_unreclaimable? yes
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056329] lowmem_reserve[]: 0 0 0 0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056331] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056338] Node 0 DMA32: 866*4kB 231*8kB 18*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7776kB
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056344] 1427 total pagecache pages
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056346] 0 pages in swap cache
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056347] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056348] Free swap = 0kB
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.056349] Total swap = 0kB
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061569] 985072 pages RAM
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061572] 24998 pages reserved
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061573] 3793 pages shared
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061574] 953266 pages non-shared
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061576] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061585] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061589] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061592] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061595] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061598] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061601] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061605] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061608] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061612] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061615] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061618] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061622] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061625] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061628] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061631] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061634] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061637] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061640] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061643] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061647] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061650] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061653] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061656] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061659] [13468] 0 13468 89584 2502 0 0 0 php5-fpm
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061662] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061665] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061668] [ 2686] 1012 2686 68618 5660 0 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061672] [ 8098] 108 8098 6792 76 0 0 0 pickup
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061675] [30806] 1012 30806 888684 825558 0 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061678] [ 710] 1012 710 68580 5605 1 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061681] [ 847] 1012 847 1099 25 1 0 0 sh
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061684] [ 848] 1012 848 65963 3547 1 0 0 php
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061687] Out of memory: Kill process 30806 (php) score 861 or sacrifice child
Jul 15 10:25:54 ip-10-61-174-46 kernel: [1410850.061699] Killed process 30806 (php) total-vm:3554736kB, anon-rss:3302232kB, file-rss:0kB
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876165] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876168] php cpuset=/ mems_allowed=0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876171] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876173] Call Trace:
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876181] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876184] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876187] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876190] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876197] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876205] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876209] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876212] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876216] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876219] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876223] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876226] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876229] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876233] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876236] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876239] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876242] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876247] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876251] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876255] Mem-Info:
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876256] Node 0 DMA per-cpu:
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876259] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876261] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876262] Node 0 DMA32 per-cpu:
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876265] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876267] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876270] active_anon:933073 inactive_anon:1393 isolated_anon:0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876271] active_file:3 inactive_file:17 isolated_file:0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876272] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876273] free:4048 slab_reclaimable:2516 slab_unreclaimable:2578
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876274] mapped:1343 shmem:1405 pagetables:3072 bounce:0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876276] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876284] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876288] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732292kB inactive_anon:5572kB active_file:12kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5372kB shmem:5620kB slab_reclaimable:10064kB slab_unreclaimable:10312kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24227 all_unreclaimable? yes
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876297] lowmem_reserve[]: 0 0 0 0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876300] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876309] Node 0 DMA32: 672*4kB 168*8kB 58*16kB 24*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876318] 1437 total pagecache pages
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876320] 0 pages in swap cache
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876322] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876323] Free swap = 0kB
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.876325] Total swap = 0kB
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882161] 985072 pages RAM
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882163] 24998 pages reserved
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882165] 3799 pages shared
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882166] 953331 pages non-shared
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882168] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882176] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882180] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882183] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882186] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882189] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882193] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882196] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882199] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882203] [ 790] 101 790 63477 630 1 0 0 rsyslogd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882207] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882210] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882213] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882216] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882222] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882225] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882228] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882231] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882234] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882237] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882240] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882243] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882246] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882249] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882252] [13468] 0 13468 89584 2525 1 0 0 php5-fpm
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882256] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882259] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882262] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882266] [ 8098] 108 8098 6792 76 1 0 0 pickup
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882269] [ 880] 1012 880 892207 829116 0 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882272] [ 3125] 1012 3125 68580 5603 0 0 0 php
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882275] Out of memory: Kill process 880 (php) score 865 or sacrifice child
Jul 15 10:29:49 ip-10-61-174-46 kernel: [1411084.882289] Killed process 880 (php) total-vm:3568828kB, anon-rss:3316376kB, file-rss:88kB
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008505] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008510] php cpuset=/ mems_allowed=0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008512] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008514] Call Trace:
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008521] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008523] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008526] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008529] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008532] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008536] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008540] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008542] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008545] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008548] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008551] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008553] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008556] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008558] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008561] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008563] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008565] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008568] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008573] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008575] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008578] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008581] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008582] Mem-Info:
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008583] Node 0 DMA per-cpu:
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008585] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008587] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008588] Node 0 DMA32 per-cpu:
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008589] CPU 0: hi: 186, btch: 31 usd: 39
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008591] CPU 1: hi: 186, btch: 31 usd: 30
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008593] active_anon:933083 inactive_anon:1393 isolated_anon:0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008594] active_file:0 inactive_file:0 isolated_file:0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008594] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008595] free:4044 slab_reclaimable:2517 slab_unreclaimable:2575
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008596] mapped:1355 shmem:1405 pagetables:3077 bounce:0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008597] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008604] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008607] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3732332kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10068kB slab_unreclaimable:10300kB kernel_stack:728kB pagetables:12308kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:85 all_unreclaimable? yes
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008613] lowmem_reserve[]: 0 0 0 0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008616] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008623] Node 0 DMA32: 890*4kB 193*8kB 33*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008630] 1465 total pagecache pages
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008631] 0 pages in swap cache
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008632] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008633] Free swap = 0kB
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.008634] Total swap = 0kB
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013938] 985072 pages RAM
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013940] 24998 pages reserved
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013941] 3783 pages shared
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013943] 953289 pages non-shared
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013945] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013954] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013957] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013960] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013963] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013967] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013970] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013973] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013977] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013981] [ 790] 101 790 63477 630 1 0 0 rsyslogd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013984] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013987] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013991] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013994] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.013997] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014000] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014003] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014006] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014009] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014013] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014016] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014019] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014022] [ 658] 1012 658 134393 70798 1 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014025] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014028] [13468] 0 13468 89584 2541 0 0 0 php5-fpm
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014031] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014034] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014037] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014041] [ 8098] 108 8098 6792 76 1 0 0 pickup
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014044] [ 3257] 1012 3257 892243 829129 0 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014047] [ 5015] 1012 5015 68580 5603 0 0 0 php
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014050] Out of memory: Kill process 3257 (php) score 865 or sacrifice child
Jul 15 10:32:46 ip-10-61-174-46 kernel: [1411262.014061] Killed process 3257 (php) total-vm:3568972kB, anon-rss:3316516kB, file-rss:0kB
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120144] php cpuset=/ mems_allowed=0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120147] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120148] Call Trace:
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120155] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120157] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120159] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120162] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120167] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120173] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120175] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120181] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120184] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120186] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120188] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120191] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120193] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120196] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120202] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120205] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120210] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120213] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120215] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120218] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120220] Mem-Info:
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120221] Node 0 DMA per-cpu:
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120223] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120225] Node 0 DMA32 per-cpu:
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120227] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120228] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120231] active_anon:933072 inactive_anon:1393 isolated_anon:0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120232] active_file:0 inactive_file:0 isolated_file:0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120232] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120233] free:4048 slab_reclaimable:2576 slab_unreclaimable:2573
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120234] mapped:1366 shmem:1405 pagetables:3073 bounce:0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120242] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120245] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732288kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:10304kB slab_unreclaimable:10292kB kernel_stack:728kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120251] lowmem_reserve[]: 0 0 0 0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120260] Node 0 DMA32: 986*4kB 221*8kB 3*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120267] 1477 total pagecache pages
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120268] 0 pages in swap cache
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120269] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120270] Free swap = 0kB
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.120271] Total swap = 0kB
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125675] 985072 pages RAM
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125678] 24998 pages reserved
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125679] 3817 pages shared
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125680] 953319 pages non-shared
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125682] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125690] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125694] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125697] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125700] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125703] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125706] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125710] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125713] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125717] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125720] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125724] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125727] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125730] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125733] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125736] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125739] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125742] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125745] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125749] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125752] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125755] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125758] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125761] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125764] [13468] 0 13468 89584 2522 1 0 0 php5-fpm
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125767] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125770] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125773] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125777] [ 8098] 108 8098 6792 76 0 0 0 pickup
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125780] [ 5182] 1012 5182 892243 829130 0 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125783] [14326] 1012 14326 68580 5604 1 0 0 php
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125786] Out of memory: Kill process 5182 (php) score 865 or sacrifice child
Jul 15 10:48:23 ip-10-61-174-46 kernel: [1412199.125798] Killed process 5182 (php) total-vm:3568972kB, anon-rss:3316520kB, file-rss:0kB
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012471] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012475] php cpuset=/ mems_allowed=0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012478] Pid: 14571, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012480] Call Trace:
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012487] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012489] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012491] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012494] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012499] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012502] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012505] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012507] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012510] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012513] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012515] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012519] [<ffffffff81006337>] ? pte_mfn_to_pfn+0x67/0x110
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012521] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012523] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012526] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012528] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012531] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012533] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012535] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012538] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012540] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012543] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012545] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012547] Mem-Info:
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012548] Node 0 DMA per-cpu:
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012550] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012551] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012552] Node 0 DMA32 per-cpu:
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012554] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012557] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012559] active_anon:933405 inactive_anon:1393 isolated_anon:0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012560] active_file:20 inactive_file:10 isolated_file:0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012561] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012561] free:4048 slab_reclaimable:2263 slab_unreclaimable:2580
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012562] mapped:1358 shmem:1405 pagetables:3068 bounce:0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012564] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012570] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012573] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733620kB inactive_anon:5572kB active_file:80kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9052kB slab_unreclaimable:10320kB kernel_stack:728kB pagetables:12272kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:192 all_unreclaimable? yes
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012580] lowmem_reserve[]: 0 0 0 0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012582] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012589] Node 0 DMA32: 551*4kB 289*8kB 70*16kB 10*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012596] 1430 total pagecache pages
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012597] 0 pages in swap cache
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012598] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012599] Free swap = 0kB
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.012600] Total swap = 0kB
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017770] 985072 pages RAM
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017772] 24998 pages reserved
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017774] 3775 pages shared
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017775] 953325 pages non-shared
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017777] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017786] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017789] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017792] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017795] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017798] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017802] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017805] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017809] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017813] [ 790] 101 790 63477 630 1 0 0 rsyslogd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017816] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017819] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017822] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017825] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017828] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017831] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017834] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017837] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017840] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017844] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017847] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017850] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017853] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017856] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017859] [13468] 0 13468 89584 2507 1 0 0 php5-fpm
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017862] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017865] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017869] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017872] [ 8098] 108 8098 6792 76 0 0 0 pickup
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017876] [14571] 1012 14571 892563 829462 0 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017879] [14704] 1012 14704 68580 5604 1 0 0 php
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017881] Out of memory: Kill process 14571 (php) score 865 or sacrifice child
Jul 15 10:48:50 ip-10-61-174-46 kernel: [1412226.017892] Killed process 14571 (php) total-vm:3570252kB, anon-rss:3317848kB, file-rss:0kB
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584177] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584181] php cpuset=/ mems_allowed=0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584184] Pid: 13231, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584186] Call Trace:
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584193] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584195] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584198] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584200] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584204] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584208] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584210] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584213] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584215] [<ffffffff8113c9e8>] ? do_wp_page+0x2e8/0x740
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584219] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584221] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584223] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584226] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584228] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584232] [<ffffffff8105761b>] ? check_preempt_wakeup+0x15b/0x230
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584236] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584239] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584242] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584245] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584248] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584250] Mem-Info:
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584251] Node 0 DMA per-cpu:
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584253] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584254] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584255] Node 0 DMA32 per-cpu:
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584257] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584258] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584261] active_anon:932762 inactive_anon:1393 isolated_anon:0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584261] active_file:30 inactive_file:32 isolated_file:0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584262] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584263] free:4048 slab_reclaimable:2605 slab_unreclaimable:2611
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584263] mapped:1369 shmem:1405 pagetables:3219 bounce:0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584265] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584272] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584275] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731048kB inactive_anon:5572kB active_file:120kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:10420kB slab_unreclaimable:10444kB kernel_stack:768kB pagetables:12876kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14048 all_unreclaimable? yes
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584281] lowmem_reserve[]: 0 0 0 0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584284] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584290] Node 0 DMA32: 843*4kB 155*8kB 48*16kB 8*32kB 4*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584297] 1515 total pagecache pages
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584298] 0 pages in swap cache
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584300] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584301] Free swap = 0kB
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.584302] Total swap = 0kB
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589859] 985072 pages RAM
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589862] 24998 pages reserved
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589863] 3829 pages shared
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589865] 953287 pages non-shared
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589867] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589875] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589879] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589882] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589885] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589888] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589891] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589894] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589898] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589902] [ 790] 101 790 63477 630 0 0 0 rsyslogd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589905] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589908] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589912] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589915] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589918] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589921] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589924] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589927] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589930] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589934] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589937] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589940] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589943] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589946] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589949] [13468] 0 13468 89584 2519 0 0 0 php5-fpm
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589952] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589955] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589958] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589962] [ 2077] 108 2077 6792 69 1 0 0 pickup
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589965] [ 6968] 1012 6968 885843 822684 0 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589968] [13000] 1012 13000 68580 5603 0 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589971] [13230] 1012 13230 1099 25 1 0 0 sh
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589974] [13231] 1012 13231 68825 5872 1 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589977] [13237] 1012 13237 1099 25 1 0 0 sh
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589980] [13238] 1012 13238 14040 63 1 0 0 git
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589983] [13239] 1012 13239 12484 152 0 0 0 php
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589985] Out of memory: Kill process 6968 (php) score 858 or sacrifice child
Jul 15 11:44:07 ip-10-61-174-46 kernel: [1415542.589996] Killed process 6968 (php) total-vm:3543372kB, anon-rss:3290628kB, file-rss:108kB
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548117] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548121] php cpuset=/ mems_allowed=0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548124] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548126] Call Trace:
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548133] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548135] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548137] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548140] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548145] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548148] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548154] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548157] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548161] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548163] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548166] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548169] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548171] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548174] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548176] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548179] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548181] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548185] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548188] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548190] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548192] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548194] Mem-Info:
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548195] Node 0 DMA per-cpu:
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548197] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548198] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548199] Node 0 DMA32 per-cpu:
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548201] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548202] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548205] active_anon:933026 inactive_anon:1393 isolated_anon:0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548206] active_file:0 inactive_file:0 isolated_file:0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548206] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548207] free:4048 slab_reclaimable:2495 slab_unreclaimable:2619
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548208] mapped:1381 shmem:1405 pagetables:3053 bounce:0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548209] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548216] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548219] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732104kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:9980kB slab_unreclaimable:10476kB kernel_stack:728kB pagetables:12212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548225] lowmem_reserve[]: 0 0 0 0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548228] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548234] Node 0 DMA32: 796*4kB 210*8kB 49*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548241] 1448 total pagecache pages
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548242] 0 pages in swap cache
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548244] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548245] Free swap = 0kB
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.548246] Total swap = 0kB
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553451] 985072 pages RAM
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553453] 24998 pages reserved
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553454] 3795 pages shared
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553456] 953344 pages non-shared
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553457] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553466] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553470] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553473] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553476] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553479] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553483] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553486] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553489] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553493] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553496] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553499] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553503] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553506] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553509] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553512] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553515] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553518] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553521] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553524] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553528] [ 654] 1012 654 70351 5759 0 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553530] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553533] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553536] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553539] [13468] 0 13468 89584 2504 0 0 0 php5-fpm
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553543] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553546] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553549] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553552] [ 2077] 108 2077 6792 69 1 0 0 pickup
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553556] [13283] 1012 13283 892243 829097 0 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553559] [14187] 1012 14187 68580 5604 0 0 0 php
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553562] Out of memory: Kill process 13283 (php) score 865 or sacrifice child
Jul 15 11:46:08 ip-10-61-174-46 kernel: [1415663.553572] Killed process 13283 (php) total-vm:3568972kB, anon-rss:3316388kB, file-rss:0kB
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032169] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032174] php cpuset=/ mems_allowed=0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032177] Pid: 14401, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032179] Call Trace:
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032186] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032189] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032191] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032194] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032199] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032202] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032206] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032208] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032211] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032214] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032216] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032219] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032221] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032224] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032227] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032229] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032232] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032235] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032237] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032240] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032242] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032244] Mem-Info:
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032245] Node 0 DMA per-cpu:
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032247] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032248] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032250] Node 0 DMA32 per-cpu:
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032251] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032253] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032255] active_anon:933043 inactive_anon:1393 isolated_anon:0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032256] active_file:0 inactive_file:0 isolated_file:0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032257] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032257] free:4048 slab_reclaimable:2517 slab_unreclaimable:2581
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032258] mapped:1362 shmem:1405 pagetables:3071 bounce:0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032259] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032266] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032269] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732172kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:10068kB slab_unreclaimable:10324kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032275] lowmem_reserve[]: 0 0 0 0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032278] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032285] Node 0 DMA32: 860*4kB 156*8kB 48*16kB 11*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032291] 1438 total pagecache pages
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032292] 0 pages in swap cache
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032294] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032295] Free swap = 0kB
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.032296] Total swap = 0kB
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037521] 985072 pages RAM
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037524] 24998 pages reserved
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037525] 3816 pages shared
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037526] 953309 pages non-shared
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037528] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037537] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037540] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037543] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037546] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037549] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037553] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037556] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037559] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037563] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037567] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037570] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037573] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037576] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037579] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037582] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037585] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037588] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037591] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037595] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037597] [ 654] 1012 654 70351 5759 1 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037600] [ 657] 1012 657 69242 6260 1 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037603] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037606] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037609] [13468] 0 13468 89584 2533 0 0 0 php5-fpm
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037612] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037615] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037618] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037622] [ 2077] 108 2077 6792 70 1 0 0 pickup
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037625] [14401] 1012 14401 892243 829095 0 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037628] [17552] 1012 17552 68580 5606 0 0 0 php
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037631] Out of memory: Kill process 14401 (php) score 865 or sacrifice child
Jul 15 11:52:27 ip-10-61-174-46 kernel: [1416043.037642] Killed process 14401 (php) total-vm:3568972kB, anon-rss:3316380kB, file-rss:0kB
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840149] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840153] php cpuset=/ mems_allowed=0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840155] Pid: 17625, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840157] Call Trace:
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840163] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840166] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840168] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840171] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840176] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840178] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840182] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840184] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840187] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840190] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840193] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840195] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840198] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840200] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840203] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840206] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840208] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840210] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840214] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840215] Mem-Info:
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840216] Node 0 DMA per-cpu:
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840218] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840220] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840221] Node 0 DMA32 per-cpu:
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840223] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840224] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840226] active_anon:932922 inactive_anon:1393 isolated_anon:0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840227] active_file:0 inactive_file:4 isolated_file:0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840228] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840228] free:4048 slab_reclaimable:2512 slab_unreclaimable:2611
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840229] mapped:1361 shmem:1405 pagetables:3169 bounce:0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840231] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840237] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840240] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731688kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10048kB slab_unreclaimable:10444kB kernel_stack:744kB pagetables:12676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2697 all_unreclaimable? yes
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840247] lowmem_reserve[]: 0 0 0 0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840249] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840256] Node 0 DMA32: 811*4kB 165*8kB 57*16kB 14*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7972kB
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840262] 1460 total pagecache pages
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840264] 0 pages in swap cache
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840265] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840266] Free swap = 0kB
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.840267] Total swap = 0kB
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845437] 985072 pages RAM
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845439] 24998 pages reserved
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845440] 3787 pages shared
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845441] 953319 pages non-shared
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845443] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845452] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845456] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845459] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845462] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845465] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845468] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845472] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845475] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845479] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845482] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845485] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845488] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845491] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845494] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845498] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845501] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845504] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845507] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845510] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845513] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845516] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845519] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845522] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845525] [13468] 0 13468 89584 2503 0 0 0 php5-fpm
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845529] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845532] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845535] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845538] [ 2077] 108 2077 6792 70 0 0 0 pickup
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845542] [17625] 1012 17625 889363 826275 0 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845545] [19378] 1012 19378 68580 5605 1 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845548] [19607] 1012 19607 1099 24 1 0 0 sh
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845551] [19608] 1012 19608 65118 2673 1 0 0 php
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845553] Out of memory: Kill process 17625 (php) score 862 or sacrifice child
Jul 15 11:56:18 ip-10-61-174-46 kernel: [1416273.845564] Killed process 17625 (php) total-vm:3557452kB, anon-rss:3305100kB, file-rss:0kB
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668144] php cpuset=/ mems_allowed=0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668147] Pid: 19640, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668149] Call Trace:
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668156] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668158] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668160] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668170] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668172] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668175] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668178] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668180] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668184] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668186] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668188] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668192] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668194] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668196] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668199] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668202] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668203] Mem-Info:
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668205] Node 0 DMA per-cpu:
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668207] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668208] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668209] Node 0 DMA32 per-cpu:
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668211] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668212] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668215] active_anon:932940 inactive_anon:1393 isolated_anon:0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668215] active_file:0 inactive_file:13 isolated_file:0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668216] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668216] free:4048 slab_reclaimable:2417 slab_unreclaimable:2591
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668217] mapped:1336 shmem:1405 pagetables:3262 bounce:0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668219] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668225] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668228] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731760kB inactive_anon:5572kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5344kB shmem:5620kB slab_reclaimable:9668kB slab_unreclaimable:10364kB kernel_stack:736kB pagetables:13048kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:848 all_unreclaimable? yes
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668235] lowmem_reserve[]: 0 0 0 0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668237] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668244] Node 0 DMA32: 826*4kB 293*8kB 16*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668251] 1413 total pagecache pages
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668252] 0 pages in swap cache
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668253] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668254] Free swap = 0kB
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.668255] Total swap = 0kB
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673486] 985072 pages RAM
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673489] 24998 pages reserved
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673490] 3777 pages shared
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673491] 953331 pages non-shared
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673493] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673501] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673505] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673508] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673511] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673514] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673518] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673521] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673524] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673528] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673531] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673534] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673538] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673541] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673544] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673550] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673553] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673557] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673560] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673563] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673566] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673569] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673573] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673575] [13468] 0 13468 89584 2541 1 0 0 php5-fpm
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673578] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673581] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673585] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673588] [ 2077] 108 2077 6792 70 1 0 0 pickup
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673591] [19640] 1012 19640 885676 822541 0 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673595] [ 2845] 1012 2845 68580 5605 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673598] [ 2997] 1012 2997 1099 24 1 0 0 sh
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673601] [ 2998] 1012 2998 68825 5870 1 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673604] [ 3004] 1012 3004 1099 25 1 0 0 sh
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673607] [ 3005] 1012 3005 4703 67 1 0 0 git
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673610] [ 3006] 1012 3006 39759 454 0 0 0 php
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673613] Out of memory: Kill process 19640 (php) score 858 or sacrifice child
Jul 15 12:28:19 ip-10-61-174-46 kernel: [1418194.673624] Killed process 19640 (php) total-vm:3542704kB, anon-rss:3290164kB, file-rss:0kB
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844128] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844132] php5-fpm cpuset=/ mems_allowed=0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844135] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844136] Call Trace:
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844143] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844146] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844148] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844152] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844155] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844159] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844162] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844165] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844168] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844172] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844174] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844177] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844180] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844182] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844185] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844187] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844191] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844195] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844199] [<ffffffff8117ae40>] ? fget+0x30/0x90
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844201] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844203] Mem-Info:
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844204] Node 0 DMA per-cpu:
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844206] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844207] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844208] Node 0 DMA32 per-cpu:
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844210] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844211] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844214] active_anon:933169 inactive_anon:1393 isolated_anon:0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844214] active_file:10 inactive_file:7 isolated_file:0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844215] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844216] free:4048 slab_reclaimable:2422 slab_unreclaimable:2587
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844216] mapped:1377 shmem:1405 pagetables:3072 bounce:0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844218] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844224] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844227] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732676kB inactive_anon:5572kB active_file:40kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:9688kB slab_unreclaimable:10348kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12675 all_unreclaimable? yes
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844234] lowmem_reserve[]: 0 0 0 0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844236] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844243] Node 0 DMA32: 741*4kB 172*8kB 44*16kB 23*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7956kB
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844249] 1449 total pagecache pages
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844251] 0 pages in swap cache
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844252] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844253] Free swap = 0kB
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.844254] Total swap = 0kB
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849454] 985072 pages RAM
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849457] 24998 pages reserved
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849458] 3775 pages shared
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849459] 953337 pages non-shared
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849461] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849470] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849473] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849476] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849480] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849483] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849486] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849489] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849493] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849496] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849500] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849503] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849506] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849509] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849512] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849515] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849518] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849522] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849525] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849528] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849531] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849534] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849537] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849540] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849543] [13468] 0 13468 89584 2526 1 0 0 php5-fpm
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849546] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849549] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849553] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849556] [ 2077] 108 2077 6792 70 1 0 0 pickup
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849560] [ 3130] 1012 3130 892307 829200 0 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849562] [ 3712] 1012 3712 68580 5604 1 0 0 php
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849565] Out of memory: Kill process 3130 (php) score 865 or sacrifice child
Jul 15 12:29:50 ip-10-61-174-46 kernel: [1418285.849576] Killed process 3130 (php) total-vm:3569228kB, anon-rss:3316796kB, file-rss:4kB
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404148] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404153] php cpuset=/ mems_allowed=0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404155] Pid: 3884, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404157] Call Trace:
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404164] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404166] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404168] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404171] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404174] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404178] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404180] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404183] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404186] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404191] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404194] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404196] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404202] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404204] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404207] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404210] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404211] Mem-Info:
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404212] Node 0 DMA per-cpu:
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404214] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404217] Node 0 DMA32 per-cpu:
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404222] active_anon:933160 inactive_anon:1393 isolated_anon:0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404223] active_file:0 inactive_file:0 isolated_file:0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404223] unevictable:0 dirty:0 writeback:37 unstable:0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404224] free:4048 slab_reclaimable:2503 slab_unreclaimable:2587
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404225] mapped:1333 shmem:1405 pagetables:3074 bounce:0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404233] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732640kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:148kB mapped:5332kB shmem:5620kB slab_reclaimable:10012kB slab_unreclaimable:10348kB kernel_stack:736kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404242] lowmem_reserve[]: 0 0 0 0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404252] Node 0 DMA32: 1057*4kB 195*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404258] 1436 total pagecache pages
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404260] 0 pages in swap cache
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404261] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404262] Free swap = 0kB
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.404263] Total swap = 0kB
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409418] 985072 pages RAM
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409421] 24998 pages reserved
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409422] 3790 pages shared
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409423] 953339 pages non-shared
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409425] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409433] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409437] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409440] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409443] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409446] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409450] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409453] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409456] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409461] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409464] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409467] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409470] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409473] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409476] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409479] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409482] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409486] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409489] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409492] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409495] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409498] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409501] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409505] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409508] [13468] 0 13468 89584 2511 1 0 0 php5-fpm
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409511] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409514] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409517] [ 2686] 1012 2686 68618 5660 0 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409521] [ 2077] 108 2077 6792 70 0 0 0 pickup
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409524] [ 3884] 1012 3884 892371 829200 1 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409527] [ 5195] 1012 5195 68580 5605 1 0 0 php
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409530] Out of memory: Kill process 3884 (php) score 865 or sacrifice child
Jul 15 12:32:46 ip-10-61-174-46 kernel: [1418462.409542] Killed process 3884 (php) total-vm:3569484kB, anon-rss:3316800kB, file-rss:0kB
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817823] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817826] php cpuset=/ mems_allowed=0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817829] Pid: 5352, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817831] Call Trace:
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817838] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817840] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817842] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817846] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817849] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817852] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817854] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817857] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817860] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817863] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817866] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817868] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817871] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817873] [<ffffffff8113f9d1>] ? handle_mm_fault+0x1/0x370
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817876] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817879] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817882] [<ffffffff8165c2f1>] ? __schedule+0x6b1/0x6f0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817884] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817886] Mem-Info:
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817887] Node 0 DMA per-cpu:
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817889] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817890] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817891] Node 0 DMA32 per-cpu:
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817893] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817894] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817897] active_anon:933229 inactive_anon:1393 isolated_anon:0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817898] active_file:15 inactive_file:10 isolated_file:0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817898] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817899] free:4048 slab_reclaimable:2468 slab_unreclaimable:2541
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817900] mapped:1371 shmem:1405 pagetables:3076 bounce:0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817901] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817908] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817911] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732916kB inactive_anon:5572kB active_file:60kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5484kB shmem:5620kB slab_reclaimable:9872kB slab_unreclaimable:10164kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:157 all_unreclaimable? yes
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817917] lowmem_reserve[]: 0 0 0 0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817920] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817926] Node 0 DMA32: 799*4kB 170*8kB 63*16kB 11*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7964kB
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817933] 1441 total pagecache pages
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817934] 0 pages in swap cache
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817935] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817936] Free swap = 0kB
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.817937] Total swap = 0kB
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823149] 985072 pages RAM
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823151] 24998 pages reserved
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823152] 3773 pages shared
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823154] 953337 pages non-shared
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823155] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823164] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823168] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823171] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823174] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823177] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823180] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823184] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823187] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823191] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823194] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823197] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823200] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823203] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823207] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823210] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823213] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823216] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823219] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823222] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823226] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823229] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823232] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823235] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823238] [13468] 0 13468 89584 2496 1 0 0 php5-fpm
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823241] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823244] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823248] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823251] [ 2077] 108 2077 6792 70 0 0 0 pickup
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823254] [ 5352] 1012 5352 892371 829235 0 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823258] [12198] 1012 12198 68580 5606 1 0 0 php
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823260] Out of memory: Kill process 5352 (php) score 865 or sacrifice child
Jul 15 12:47:40 ip-10-61-174-46 kernel: [1419355.823272] Killed process 5352 (php) total-vm:3569484kB, anon-rss:3316940kB, file-rss:0kB
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916139] php cpuset=/ mems_allowed=0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916143] Pid: 12322, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916145] Call Trace:
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916160] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916167] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916172] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916175] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916179] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916184] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916187] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916190] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916194] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916196] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916198] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916204] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916206] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916208] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916211] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916213] Mem-Info:
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916214] Node 0 DMA per-cpu:
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916216] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916217] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916218] Node 0 DMA32 per-cpu:
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916220] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916221] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916224] active_anon:932617 inactive_anon:1393 isolated_anon:0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916224] active_file:0 inactive_file:0 isolated_file:0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916225] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916226] free:4031 slab_reclaimable:2346 slab_unreclaimable:2617
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916226] mapped:1366 shmem:1405 pagetables:3553 bounce:0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916235] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916237] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3730468kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9384kB slab_unreclaimable:10468kB kernel_stack:840kB pagetables:14212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916244] lowmem_reserve[]: 0 0 0 0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916246] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916253] Node 0 DMA32: 627*4kB 201*8kB 47*16kB 6*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916260] 1471 total pagecache pages
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916261] 0 pages in swap cache
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916262] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916263] Free swap = 0kB
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.916264] Total swap = 0kB
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921526] 985072 pages RAM
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921528] 24998 pages reserved
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921529] 3789 pages shared
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921530] 953345 pages non-shared
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921532] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921541] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921545] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921548] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921551] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921554] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921557] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921561] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921564] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921568] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921571] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921574] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921577] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921580] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921584] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921587] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921591] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921594] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921598] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921601] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921604] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921607] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921610] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921613] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921616] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921619] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921622] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921625] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921629] [ 2077] 108 2077 6792 70 0 0 0 pickup
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921632] [12322] 1012 12322 868370 805176 0 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921635] [12428] 1012 12428 68580 5605 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921638] [12570] 1012 12570 1099 24 1 0 0 sh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921641] [12571] 1012 12571 1099 25 1 0 0 sh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921645] [12572] 1012 12572 68825 5870 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921648] [12573] 1012 12573 68825 5872 0 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921651] [12584] 1012 12584 1099 26 1 0 0 sh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921654] [12585] 1012 12585 1099 25 1 0 0 sh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921657] [12586] 1012 12586 4675 60 1 0 0 git
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921660] [12587] 1012 12587 4676 58 1 0 0 git
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921663] [12588] 1012 12588 68499 5552 1 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921666] [12589] 1012 12589 68499 5551 0 0 0 php
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921669] [12592] 1012 12592 1099 25 1 0 0 sh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921672] [12593] 1012 12593 10901 176 1 0 0 ssh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921675] [12594] 1012 12594 1099 24 0 0 0 sh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921678] [12595] 1012 12595 10901 175 1 0 0 ssh
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921680] Out of memory: Kill process 12322 (php) score 840 or sacrifice child
Jul 15 12:48:24 ip-10-61-174-46 kernel: [1419399.921692] Killed process 12322 (php) total-vm:3473480kB, anon-rss:3220704kB, file-rss:0kB
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271343] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271347] php5-fpm cpuset=/ mems_allowed=0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271350] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271351] Call Trace:
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271358] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271361] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271363] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271366] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271369] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271373] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271375] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271378] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271382] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271384] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271387] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271390] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271392] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271395] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271398] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271403] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271407] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271410] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271411] Mem-Info:
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271413] Node 0 DMA per-cpu:
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271414] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271416] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271417] Node 0 DMA32 per-cpu:
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271419] CPU 0: hi: 186, btch: 31 usd: 83
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271420] CPU 1: hi: 186, btch: 31 usd: 14
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271423] active_anon:933249 inactive_anon:1393 isolated_anon:0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271423] active_file:24 inactive_file:22 isolated_file:8
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271424] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271425] free:4029 slab_reclaimable:2361 slab_unreclaimable:2549
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271425] mapped:1356 shmem:1405 pagetables:3075 bounce:0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271427] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271434] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271437] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3732996kB inactive_anon:5572kB active_file:96kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9444kB slab_unreclaimable:10196kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:411 all_unreclaimable? yes
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271443] lowmem_reserve[]: 0 0 0 0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271446] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271452] Node 0 DMA32: 455*4kB 97*8kB 146*16kB 21*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7780kB
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271459] 1430 total pagecache pages
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271460] 0 pages in swap cache
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271462] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271463] Free swap = 0kB
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.271464] Total swap = 0kB
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276632] 985072 pages RAM
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276634] 24998 pages reserved
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276635] 3779 pages shared
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276636] 953280 pages non-shared
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276638] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276647] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276651] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276654] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276657] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276660] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276663] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276667] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276670] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276674] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276677] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276680] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276684] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276687] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276690] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276693] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276696] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276700] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276703] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276706] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276709] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276712] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276715] [ 658] 1012 658 134393 70798 1 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276718] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276721] [13468] 0 13468 89584 2527 0 0 0 php5-fpm
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276724] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276727] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276731] [ 2686] 1012 2686 68618 5660 1 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276734] [ 2077] 108 2077 6792 70 0 0 0 pickup
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276738] [12691] 1012 12691 892435 829298 1 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276741] [12693] 1012 12693 68580 5604 0 0 0 php
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276744] Out of memory: Kill process 12691 (php) score 865 or sacrifice child
Jul 15 12:48:52 ip-10-61-174-46 kernel: [1419428.276755] Killed process 12691 (php) total-vm:3569740kB, anon-rss:3317192kB, file-rss:0kB
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504133] php cpuset=/ mems_allowed=0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504136] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504138] Call Trace:
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504145] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504147] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504149] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504152] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504157] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504160] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504164] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504166] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504169] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504171] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504175] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504177] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504179] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504182] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504184] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504188] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504191] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504195] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504197] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504200] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504202] Mem-Info:
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504203] Node 0 DMA per-cpu:
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504205] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504206] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504207] Node 0 DMA32 per-cpu:
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504209] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504210] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504212] active_anon:932892 inactive_anon:1393 isolated_anon:0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504213] active_file:0 inactive_file:0 isolated_file:0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504214] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504214] free:4048 slab_reclaimable:2685 slab_unreclaimable:2565
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504215] mapped:1345 shmem:1405 pagetables:3084 bounce:0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504217] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504223] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731568kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5380kB shmem:5620kB slab_reclaimable:10740kB slab_unreclaimable:10260kB kernel_stack:728kB pagetables:12336kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504233] lowmem_reserve[]: 0 0 0 0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504235] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504242] Node 0 DMA32: 929*4kB 251*8kB 10*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504249] 1486 total pagecache pages
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504250] 0 pages in swap cache
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504251] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504252] Free swap = 0kB
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.504253] Total swap = 0kB
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509337] 985072 pages RAM
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509340] 24998 pages reserved
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509341] 3817 pages shared
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509342] 953303 pages non-shared
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509344] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509352] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509356] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509359] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509362] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509365] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509368] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509372] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509375] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509379] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509382] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509385] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509388] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509392] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509395] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509398] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509401] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509404] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509407] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509410] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509414] [ 654] 1012 654 70351 5769 0 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509417] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509420] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509423] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509426] [13468] 0 13468 89584 2535 0 0 0 php5-fpm
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509429] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509432] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509435] [ 2686] 1012 2686 68618 5662 0 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509439] [20084] 108 20084 6792 70 0 0 0 pickup
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509442] [ 462] 1012 462 892051 828964 1 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509445] [ 951] 1012 951 68580 5604 1 0 0 php
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509448] Out of memory: Kill process 462 (php) score 865 or sacrifice child
Jul 15 13:33:37 ip-10-61-174-46 kernel: [1422113.509459] Killed process 462 (php) total-vm:3568204kB, anon-rss:3315856kB, file-rss:0kB
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628387] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628392] php cpuset=/ mems_allowed=0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628396] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628406] Call Trace:
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628414] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628418] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628421] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628424] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628430] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628434] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628438] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628441] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628445] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628448] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628452] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628455] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628458] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628463] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628467] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628470] [<ffffffff811794eb>] ? rw_verify_area+0x4b/0xf0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628472] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628474] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628477] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628478] Mem-Info:
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628480] Node 0 DMA per-cpu:
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628482] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628483] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628484] Node 0 DMA32 per-cpu:
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628486] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628487] CPU 1: hi: 186, btch: 31 usd: 33
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628490] active_anon:933101 inactive_anon:1393 isolated_anon:0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628490] active_file:6 inactive_file:25 isolated_file:0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628491] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628492] free:4031 slab_reclaimable:2502 slab_unreclaimable:2574
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628492] mapped:1366 shmem:1405 pagetables:3082 bounce:0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628494] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628500] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628503] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3732404kB inactive_anon:5572kB active_file:24kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:10008kB slab_unreclaimable:10296kB kernel_stack:728kB pagetables:12328kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:216 all_unreclaimable? yes
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628510] lowmem_reserve[]: 0 0 0 0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628513] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628519] Node 0 DMA32: 985*4kB 207*8kB 9*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628526] 1424 total pagecache pages
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628527] 0 pages in swap cache
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628528] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628530] Free swap = 0kB
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.628530] Total swap = 0kB
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633725] 985072 pages RAM
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633728] 24998 pages reserved
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633729] 3783 pages shared
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633730] 953300 pages non-shared
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633733] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633741] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633744] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633748] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633751] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633754] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633757] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633760] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633764] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633768] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633771] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633774] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633777] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633780] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633783] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633786] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633789] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633793] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633796] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633799] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633802] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633805] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633808] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633811] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633814] [13468] 0 13468 89584 2528 1 0 0 php5-fpm
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633817] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633820] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633823] [ 2686] 1012 2686 68618 5662 1 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633827] [20084] 108 20084 6792 76 1 0 0 pickup
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633831] [ 1153] 1012 1153 892264 829168 1 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633834] [ 9004] 1012 9004 68580 5604 1 0 0 php
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633837] Out of memory: Kill process 1153 (php) score 865 or sacrifice child
Jul 15 13:52:29 ip-10-61-174-46 kernel: [1423244.633847] Killed process 1153 (php) total-vm:3569056kB, anon-rss:3316672kB, file-rss:0kB
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444106] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444110] php cpuset=/ mems_allowed=0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444112] Pid: 9148, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444114] Call Trace:
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444121] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444124] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444126] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444129] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444132] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444135] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444137] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444141] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444143] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444146] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444149] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444151] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444154] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444156] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444160] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444162] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444164] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444167] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444170] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444172] Mem-Info:
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444173] Node 0 DMA per-cpu:
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444175] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444176] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444177] Node 0 DMA32 per-cpu:
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444179] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444180] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444183] active_anon:933238 inactive_anon:1393 isolated_anon:0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444183] active_file:35 inactive_file:35 isolated_file:0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444184] unevictable:0 dirty:9 writeback:0 unstable:0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444185] free:4048 slab_reclaimable:2466 slab_unreclaimable:2560
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444185] mapped:1371 shmem:1405 pagetables:3063 bounce:0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444187] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444193] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444196] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732952kB inactive_anon:5572kB active_file:140kB inactive_file:140kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:36kB writeback:0kB mapped:5484kB shmem:5620kB slab_reclaimable:9864kB slab_unreclaimable:10240kB kernel_stack:728kB pagetables:12252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:7267 all_unreclaimable? yes
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444203] lowmem_reserve[]: 0 0 0 0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444205] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444212] Node 0 DMA32: 964*4kB 230*8kB 7*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444219] 1443 total pagecache pages
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444220] 0 pages in swap cache
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444221] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444222] Free swap = 0kB
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.444223] Total swap = 0kB
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449577] 985072 pages RAM
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449580] 24998 pages reserved
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449581] 3784 pages shared
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449582] 953329 pages non-shared
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449584] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449593] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449597] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449600] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449603] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449607] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449610] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449613] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449617] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449621] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449624] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449627] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449630] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449633] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449637] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449640] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449643] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449646] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449649] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449652] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449655] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449658] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449662] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449665] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449668] [13468] 0 13468 89584 2513 0 0 0 php5-fpm
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449671] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449674] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449677] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449681] [20084] 108 20084 6792 76 0 0 0 pickup
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449684] [ 9148] 1012 9148 892435 829253 0 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449687] [10439] 1012 10439 68580 5604 0 0 0 php
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449689] Out of memory: Kill process 9148 (php) score 865 or sacrifice child
Jul 15 13:55:40 ip-10-61-174-46 kernel: [1423436.449701] Killed process 9148 (php) total-vm:3569740kB, anon-rss:3317012kB, file-rss:0kB
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780128] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780133] php cpuset=/ mems_allowed=0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780135] Pid: 10533, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780137] Call Trace:
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780144] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780146] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780148] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780152] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780155] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780158] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780160] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780163] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780166] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780169] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780172] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780174] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780177] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780180] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780182] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780184] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780187] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780189] Mem-Info:
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780191] Node 0 DMA per-cpu:
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780192] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780194] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780195] Node 0 DMA32 per-cpu:
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780196] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780198] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780200] active_anon:933306 inactive_anon:1393 isolated_anon:0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780201] active_file:0 inactive_file:0 isolated_file:0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780202] unevictable:0 dirty:20 writeback:0 unstable:0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780202] free:4048 slab_reclaimable:2369 slab_unreclaimable:2566
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780203] mapped:1368 shmem:1405 pagetables:3057 bounce:0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780205] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780211] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780214] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733224kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:80kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:9476kB slab_unreclaimable:10264kB kernel_stack:760kB pagetables:12228kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780221] lowmem_reserve[]: 0 0 0 0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780223] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780230] Node 0 DMA32: 645*4kB 116*8kB 84*16kB 24*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780237] 1437 total pagecache pages
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780238] 0 pages in swap cache
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780239] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780240] Free swap = 0kB
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.780241] Total swap = 0kB
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785487] 985072 pages RAM
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785489] 24998 pages reserved
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785490] 3803 pages shared
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785491] 953335 pages non-shared
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785493] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785502] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785505] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785508] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785511] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785514] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785518] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785521] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785524] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785528] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785531] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785534] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785538] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785541] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785544] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785550] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785553] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785556] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785560] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785563] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785566] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785569] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785572] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785575] [13468] 0 13468 89584 2498 1 0 0 php5-fpm
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785578] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785581] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785585] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785588] [20084] 108 20084 6792 76 0 0 0 pickup
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785591] [10533] 1012 10533 892499 829351 0 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785594] [10686] 1012 10686 68580 5606 1 0 0 php
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785597] Out of memory: Kill process 10533 (php) score 865 or sacrifice child
Jul 15 13:56:18 ip-10-61-174-46 kernel: [1423473.785608] Killed process 10533 (php) total-vm:3569996kB, anon-rss:3317404kB, file-rss:0kB
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904127] php cpuset=/ mems_allowed=0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904130] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904132] Call Trace:
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904140] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904152] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904155] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904159] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904161] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904164] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904167] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904169] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904172] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904174] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904176] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904181] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904184] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904187] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904190] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904193] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904195] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904198] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904200] Mem-Info:
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904201] Node 0 DMA per-cpu:
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904203] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904204] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904205] Node 0 DMA32 per-cpu:
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904207] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904208] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904210] active_anon:932999 inactive_anon:1393 isolated_anon:0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904211] active_file:0 inactive_file:7 isolated_file:0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904212] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904213] free:4048 slab_reclaimable:2498 slab_unreclaimable:2582
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904213] mapped:1354 shmem:1405 pagetables:3168 bounce:0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904215] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904221] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904224] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731996kB inactive_anon:5572kB active_file:0kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:9992kB slab_unreclaimable:10328kB kernel_stack:744kB pagetables:12672kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:697 all_unreclaimable? yes
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904231] lowmem_reserve[]: 0 0 0 0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904233] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904240] Node 0 DMA32: 692*4kB 114*8kB 87*16kB 26*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904247] 1426 total pagecache pages
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904248] 0 pages in swap cache
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904249] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904251] Free swap = 0kB
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.904251] Total swap = 0kB
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909464] 985072 pages RAM
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909466] 24998 pages reserved
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909467] 3784 pages shared
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909468] 953331 pages non-shared
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909470] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909479] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909482] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909486] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909489] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909492] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909495] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909498] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909502] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909506] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909509] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909512] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909515] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909518] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909521] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909525] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909528] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909531] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909534] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909537] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909540] [ 654] 1012 654 70351 5761 0 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909543] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909546] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909549] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909552] [13468] 0 13468 89584 2516 0 0 0 php5-fpm
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909555] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909558] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909562] [ 2686] 1012 2686 68618 5662 0 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909565] [20084] 108 20084 6792 76 0 0 0 pickup
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909568] [10810] 1012 10810 889364 826282 0 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909572] [20060] 1012 20060 68580 5604 1 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909575] [20154] 1012 20154 1099 25 1 0 0 sh
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909578] [20155] 1012 20155 65183 2725 1 0 0 php
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909581] Out of memory: Kill process 10810 (php) score 862 or sacrifice child
Jul 15 14:19:05 ip-10-61-174-46 kernel: [1424840.909592] Killed process 10810 (php) total-vm:3557456kB, anon-rss:3305128kB, file-rss:0kB
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016134] php cpuset=/ mems_allowed=0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016137] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016139] Call Trace:
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016147] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016157] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016163] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016170] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016172] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016175] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016179] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016182] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016185] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016187] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016189] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016192] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016194] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016197] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016201] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016204] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016207] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016209] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016211] Mem-Info:
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016212] Node 0 DMA per-cpu:
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016214] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016216] Node 0 DMA32 per-cpu:
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016222] active_anon:933256 inactive_anon:1393 isolated_anon:0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016223] active_file:0 inactive_file:0 isolated_file:0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016223] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016224] free:4048 slab_reclaimable:2391 slab_unreclaimable:2569
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016225] mapped:1369 shmem:1405 pagetables:3077 bounce:0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016233] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733024kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:9564kB slab_unreclaimable:10276kB kernel_stack:728kB pagetables:12308kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10 all_unreclaimable? yes
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016242] lowmem_reserve[]: 0 0 0 0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016252] Node 0 DMA32: 694*4kB 191*8kB 58*16kB 14*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016258] 1440 total pagecache pages
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016260] 0 pages in swap cache
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016261] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016262] Free swap = 0kB
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.016263] Total swap = 0kB
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021434] 985072 pages RAM
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021437] 24998 pages reserved
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021438] 3795 pages shared
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021439] 953334 pages non-shared
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021441] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021449] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021453] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021456] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021459] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021462] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021466] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021469] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021472] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021477] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021480] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021483] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021486] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021489] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021492] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021495] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021498] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021501] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021505] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021508] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021511] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021514] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021517] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021520] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021523] [13468] 0 13468 89584 2535 0 0 0 php5-fpm
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021526] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021529] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021532] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021535] [20084] 108 20084 6792 76 0 0 0 pickup
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021539] [20218] 1012 20218 892435 829287 0 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021542] [20388] 1012 20388 68580 5604 0 0 0 php
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021545] Out of memory: Kill process 20218 (php) score 865 or sacrifice child
Jul 15 14:20:00 ip-10-61-174-46 kernel: [1424896.021555] Killed process 20218 (php) total-vm:3569740kB, anon-rss:3317148kB, file-rss:0kB
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664109] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664113] php cpuset=/ mems_allowed=0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664116] Pid: 20594, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664118] Call Trace:
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664125] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664127] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664129] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664132] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664137] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664140] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664144] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664146] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664149] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664153] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664155] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664157] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664160] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664163] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664165] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664168] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664171] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664173] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664175] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664177] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664179] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664182] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664185] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664186] Mem-Info:
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664188] Node 0 DMA per-cpu:
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664189] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664191] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664192] Node 0 DMA32 per-cpu:
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664194] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664195] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664197] active_anon:933084 inactive_anon:1393 isolated_anon:0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664198] active_file:0 inactive_file:0 isolated_file:0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664199] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664199] free:4048 slab_reclaimable:2557 slab_unreclaimable:2565
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664200] mapped:1370 shmem:1405 pagetables:3056 bounce:0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664201] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664208] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664211] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732336kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:10228kB slab_unreclaimable:10260kB kernel_stack:728kB pagetables:12224kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664217] lowmem_reserve[]: 0 0 0 0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664220] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664227] Node 0 DMA32: 721*4kB 244*8kB 54*16kB 8*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664233] 1474 total pagecache pages
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664235] 0 pages in swap cache
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664236] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664237] Free swap = 0kB
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.664238] Total swap = 0kB
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669423] 985072 pages RAM
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669426] 24998 pages reserved
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669428] 3792 pages shared
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669429] 953299 pages non-shared
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669431] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669439] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669443] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669446] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669449] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669452] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669456] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669459] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669462] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669466] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669470] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669473] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669476] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669479] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669482] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669485] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669488] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669491] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669494] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669498] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669501] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669504] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669507] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669510] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669513] [13468] 0 13468 89584 2507 0 0 0 php5-fpm
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669516] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669519] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669522] [ 2686] 1012 2686 68618 5661 0 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669526] [20594] 1012 20594 892331 829148 0 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669529] [20831] 108 20831 6792 70 0 0 0 pickup
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669532] [21604] 1012 21604 68580 5605 1 0 0 php
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669534] Out of memory: Kill process 20594 (php) score 865 or sacrifice child
Jul 15 14:22:53 ip-10-61-174-46 kernel: [1425068.669546] Killed process 20594 (php) total-vm:3569324kB, anon-rss:3316592kB, file-rss:0kB
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904121] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904126] php5-fpm cpuset=/ mems_allowed=0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904130] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904132] Call Trace:
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904140] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904143] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904146] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904150] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904156] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904160] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904164] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904167] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904171] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904175] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904179] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904182] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904185] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904188] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904191] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904195] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904199] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904203] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904206] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904207] Mem-Info:
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904209] Node 0 DMA per-cpu:
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904211] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904212] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904213] Node 0 DMA32 per-cpu:
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904215] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904216] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904219] active_anon:933249 inactive_anon:1393 isolated_anon:0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904219] active_file:13 inactive_file:19 isolated_file:0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904220] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904221] free:4047 slab_reclaimable:2452 slab_unreclaimable:2525
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904221] mapped:1366 shmem:1405 pagetables:3070 bounce:0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904223] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904229] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904232] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3732996kB inactive_anon:5572kB active_file:52kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9808kB slab_unreclaimable:10100kB kernel_stack:728kB pagetables:12280kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2284 all_unreclaimable? yes
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904239] lowmem_reserve[]: 0 0 0 0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904241] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904248] Node 0 DMA32: 825*4kB 239*8kB 35*16kB 3*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7980kB
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904255] 1453 total pagecache pages
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904256] 0 pages in swap cache
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904258] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904259] Free swap = 0kB
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.904259] Total swap = 0kB
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909689] 985072 pages RAM
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909692] 24998 pages reserved
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909693] 3787 pages shared
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909694] 953322 pages non-shared
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909696] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909705] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909708] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909712] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909715] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909718] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909721] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909724] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909728] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909732] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909735] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909738] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909742] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909744] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909747] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909751] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909754] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909757] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909760] [ 1242] 107 1242 9445 164 0 0 0 ntpd
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909763] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909766] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909770] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909773] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909776] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909779] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909782] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909785] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909788] [ 2686] 1012 2686 68618 5661 0 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909792] [20831] 108 20831 6792 76 0 0 0 pickup
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909795] [21701] 1012 21701 892371 829262 1 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909798] [22850] 1012 22850 68580 5603 0 0 0 php
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909801] Out of memory: Kill process 21701 (php) score 865 or sacrifice child
Jul 15 14:25:57 ip-10-61-174-46 kernel: [1425252.909813] Killed process 21701 (php) total-vm:3569484kB, anon-rss:3317048kB, file-rss:0kB
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532134] php cpuset=/ mems_allowed=0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532138] Pid: 23032, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532140] Call Trace:
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532157] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532162] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532169] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532172] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532176] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532180] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532183] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532187] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532193] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532197] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532200] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532204] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532207] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532210] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532213] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532217] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532221] Mem-Info:
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532222] Node 0 DMA per-cpu:
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532225] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532227] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532228] Node 0 DMA32 per-cpu:
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532231] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532232] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532236] active_anon:933240 inactive_anon:1393 isolated_anon:0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532237] active_file:0 inactive_file:0 isolated_file:0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532238] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532239] free:4048 slab_reclaimable:2408 slab_unreclaimable:2534
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532240] mapped:1348 shmem:1405 pagetables:3071 bounce:0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532250] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532253] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732960kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5392kB shmem:5620kB slab_reclaimable:9632kB slab_unreclaimable:10136kB kernel_stack:720kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532262] lowmem_reserve[]: 0 0 0 0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532265] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532276] Node 0 DMA32: 803*4kB 189*8kB 49*16kB 4*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7876kB
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532286] 1458 total pagecache pages
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532288] 0 pages in swap cache
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532289] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532291] Free swap = 0kB
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.532292] Total swap = 0kB
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539490] 985072 pages RAM
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539493] 24998 pages reserved
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539494] 3787 pages shared
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539495] 953346 pages non-shared
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539496] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539505] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539509] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539512] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539514] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539517] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539520] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539523] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539526] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539530] [ 790] 101 790 63477 631 0 0 0 rsyslogd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539533] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539536] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539539] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539541] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539544] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539550] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539553] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539556] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539559] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539565] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539568] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539570] [ 658] 1012 658 134393 70798 1 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539573] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539576] [13468] 0 13468 89584 2530 1 0 0 php5-fpm
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539579] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539582] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539586] [ 2686] 1012 2686 68618 5661 0 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539590] [20831] 108 20831 6792 76 0 0 0 pickup
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539593] [23032] 1012 23032 892399 829260 0 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539596] [23920] 1012 23920 68580 5604 1 0 0 php
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539599] Out of memory: Kill process 23032 (php) score 865 or sacrifice child
Jul 15 14:28:19 ip-10-61-174-46 kernel: [1425394.539610] Killed process 23032 (php) total-vm:3569596kB, anon-rss:3317028kB, file-rss:12kB
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682835] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682840] php5-fpm cpuset=/ mems_allowed=0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682843] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682845] Call Trace:
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682853] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682855] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682857] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682860] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682863] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682867] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682869] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682872] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682876] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682880] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682882] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682885] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682887] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682890] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682892] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682894] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682899] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682903] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682905] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682907] Mem-Info:
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682908] Node 0 DMA per-cpu:
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682910] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682912] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682913] Node 0 DMA32 per-cpu:
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682914] CPU 0: hi: 186, btch: 31 usd: 95
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682916] CPU 1: hi: 186, btch: 31 usd: 12
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682918] active_anon:932785 inactive_anon:1393 isolated_anon:0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682919] active_file:0 inactive_file:4 isolated_file:14
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682920] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682920] free:4026 slab_reclaimable:2624 slab_unreclaimable:2577
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682921] mapped:1352 shmem:1405 pagetables:3238 bounce:0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682923] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682929] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682932] Node 0 DMA32 free:7768kB min:7848kB low:9808kB high:11772kB active_anon:3731140kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):56kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:10496kB slab_unreclaimable:10308kB kernel_stack:760kB pagetables:12952kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:155 all_unreclaimable? yes
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682939] lowmem_reserve[]: 0 0 0 0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682941] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682948] Node 0 DMA32: 810*4kB 228*8kB 42*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7784kB
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682955] 1431 total pagecache pages
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682956] 0 pages in swap cache
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682957] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682958] Free swap = 0kB
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.682959] Total swap = 0kB
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688076] 985072 pages RAM
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688078] 24998 pages reserved
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688079] 3779 pages shared
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688080] 953263 pages non-shared
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688082] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688091] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688095] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688098] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688101] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688104] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688108] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688111] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688114] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688118] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688121] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688124] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688128] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688131] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688134] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688137] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688140] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688143] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688146] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688150] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688153] [ 654] 1012 654 70351 5761 0 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688156] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688159] [ 658] 1012 658 134393 70799 0 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688162] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688165] [13468] 0 13468 89584 2515 0 0 0 php5-fpm
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688168] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688171] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688174] [ 2686] 1012 2686 68618 5661 1 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688178] [20831] 108 20831 6792 76 1 0 0 pickup
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688181] [23985] 1012 23985 891029 827945 0 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688184] [25060] 108 25060 6818 73 1 0 0 cleanup
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688187] [25061] 108 25061 6795 71 0 0 0 trivial-rewrite
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688190] [25062] 108 25062 11107 159 0 0 0 smtp
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688193] [25066] 108 25066 11107 158 1 0 0 smtp
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688196] [25593] 1012 25593 68580 5603 1 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688199] [25685] 1012 25685 1099 25 0 0 0 sh
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688202] [25686] 1012 25686 32613 425 0 0 0 php
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688205] Out of memory: Kill process 23985 (php) score 864 or sacrifice child
Jul 15 14:32:48 ip-10-61-174-46 kernel: [1425663.688217] Killed process 23985 (php) total-vm:3564116kB, anon-rss:3311760kB, file-rss:20kB
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133939] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133943] php cpuset=/ mems_allowed=0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133947] Pid: 25730, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133949] Call Trace:
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133956] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133960] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133963] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133966] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133970] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133974] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133977] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133981] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133986] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133991] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133994] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.133998] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134001] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134004] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134007] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134011] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134014] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134017] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134020] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134023] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134027] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134029] Mem-Info:
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134030] Node 0 DMA per-cpu:
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134033] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134034] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134036] Node 0 DMA32 per-cpu:
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134038] CPU 0: hi: 186, btch: 31 usd: 49
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134040] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134044] active_anon:933020 inactive_anon:1393 isolated_anon:0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134045] active_file:7 inactive_file:0 isolated_file:28
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134046] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134046] free:4048 slab_reclaimable:2610 slab_unreclaimable:2538
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134047] mapped:1355 shmem:1405 pagetables:3071 bounce:0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134050] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134060] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134064] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732080kB inactive_anon:5572kB active_file:28kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):112kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10440kB slab_unreclaimable:10152kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:169 all_unreclaimable? yes
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134073] lowmem_reserve[]: 0 0 0 0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134076] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134085] Node 0 DMA32: 795*4kB 217*8kB 42*16kB 6*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7892kB
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134094] 1423 total pagecache pages
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134096] 0 pages in swap cache
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134098] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134099] Free swap = 0kB
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.134101] Total swap = 0kB
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139473] 985072 pages RAM
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139476] 24998 pages reserved
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139477] 3772 pages shared
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139478] 953288 pages non-shared
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139480] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139488] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139492] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139495] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139498] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139501] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139504] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139508] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139511] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139516] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139519] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139522] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139526] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139529] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139532] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139535] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139538] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139541] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139544] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139547] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139550] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139553] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139556] [ 658] 1012 658 134393 70799 1 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139559] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139563] [13468] 0 13468 89584 2527 0 0 0 php5-fpm
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139566] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139569] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139572] [ 2686] 1012 2686 68618 5662 0 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139576] [20831] 108 20831 6792 76 1 0 0 pickup
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139579] [25730] 1012 25730 892179 829077 1 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139582] [32114] 1012 32114 68580 5604 1 0 0 php
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139585] Out of memory: Kill process 25730 (php) score 865 or sacrifice child
Jul 15 14:49:00 ip-10-61-174-46 kernel: [1426636.139596] Killed process 25730 (php) total-vm:3568716kB, anon-rss:3316308kB, file-rss:0kB
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889405] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889409] php cpuset=/ mems_allowed=0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889412] Pid: 32149, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889413] Call Trace:
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889421] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889423] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889425] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889428] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889431] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889435] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889437] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889440] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889444] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889447] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889449] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889451] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889455] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889457] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889459] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889462] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889464] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889467] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889469] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889472] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889474] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889476] Mem-Info:
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889477] Node 0 DMA per-cpu:
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889479] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889480] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889481] Node 0 DMA32 per-cpu:
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889483] CPU 0: hi: 186, btch: 31 usd: 3
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889484] CPU 1: hi: 186, btch: 31 usd: 133
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889487] active_anon:932739 inactive_anon:1393 isolated_anon:0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889488] active_file:0 inactive_file:0 isolated_file:0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889488] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889489] free:4045 slab_reclaimable:2808 slab_unreclaimable:2583
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889490] mapped:1370 shmem:1405 pagetables:3071 bounce:0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889491] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889498] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889501] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730956kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:11232kB slab_unreclaimable:10332kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:70 all_unreclaimable? yes
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889507] lowmem_reserve[]: 0 0 0 0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889510] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889516] Node 0 DMA32: 1089*4kB 170*8kB 9*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7908kB
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889523] 1439 total pagecache pages
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889524] 0 pages in swap cache
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889525] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889526] Free swap = 0kB
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.889527] Total swap = 0kB
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894896] 985072 pages RAM
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894898] 24998 pages reserved
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894899] 3790 pages shared
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894900] 953199 pages non-shared
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894902] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894911] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894915] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894918] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894921] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894924] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894928] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894931] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894934] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894938] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894941] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894944] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894948] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894951] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894954] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894957] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894960] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894963] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894967] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894970] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894973] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894976] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894979] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894982] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894985] [13468] 0 13468 89584 2512 1 0 0 php5-fpm
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894988] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894991] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894995] [ 2686] 1012 2686 68618 5664 0 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.894998] [20831] 108 20831 6792 76 1 0 0 pickup
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895001] [32149] 1012 32149 891891 828714 0 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895004] [15244] 1012 15244 68580 5605 1 0 0 php
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895007] Out of memory: Kill process 32149 (php) score 864 or sacrifice child
Jul 15 15:30:10 ip-10-61-174-46 kernel: [1429105.895019] Killed process 32149 (php) total-vm:3567564kB, anon-rss:3314856kB, file-rss:0kB
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108304] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108307] php cpuset=/ mems_allowed=0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108310] Pid: 16233, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108312] Call Trace:
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108319] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108322] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108324] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108327] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108330] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108334] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108336] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108339] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108343] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108346] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108348] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108351] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108354] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108356] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108359] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108361] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108364] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108367] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108369] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108371] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108374] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108376] Mem-Info:
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108377] Node 0 DMA per-cpu:
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108379] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108380] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108381] Node 0 DMA32 per-cpu:
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108383] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108384] CPU 1: hi: 186, btch: 31 usd: 14
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108387] active_anon:933098 inactive_anon:1393 isolated_anon:0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108387] active_file:0 inactive_file:0 isolated_file:0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108388] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108389] free:4041 slab_reclaimable:2566 slab_unreclaimable:2549
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108389] mapped:1365 shmem:1405 pagetables:3061 bounce:0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108391] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108397] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108400] Node 0 DMA32 free:7828kB min:7848kB low:9808kB high:11772kB active_anon:3732392kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:10264kB slab_unreclaimable:10196kB kernel_stack:728kB pagetables:12244kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14 all_unreclaimable? yes
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108407] lowmem_reserve[]: 0 0 0 0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108409] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108416] Node 0 DMA32: 793*4kB 216*8kB 43*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7828kB
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108422] 1421 total pagecache pages
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108424] 0 pages in swap cache
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108425] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108426] Free swap = 0kB
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.108427] Total swap = 0kB
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113621] 985072 pages RAM
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113623] 24998 pages reserved
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113624] 3790 pages shared
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113626] 953311 pages non-shared
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113627] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113636] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113640] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113643] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113646] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113649] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113652] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113655] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113659] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113663] [ 790] 101 790 63477 631 1 0 0 rsyslogd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113666] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113669] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113673] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113676] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113679] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113682] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113685] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113688] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113691] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113695] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113698] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113701] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113704] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113707] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113710] [13468] 0 13468 89584 2497 1 0 0 php5-fpm
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113713] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113716] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113719] [ 2686] 1012 2686 68618 5663 1 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113722] [20831] 108 20831 6792 76 0 0 0 pickup
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113726] [16233] 1012 16233 892243 829105 0 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113729] [16553] 1012 16553 68580 5605 1 0 0 php
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113732] Out of memory: Kill process 16233 (php) score 865 or sacrifice child
Jul 15 15:33:38 ip-10-61-174-46 kernel: [1429314.113743] Killed process 16233 (php) total-vm:3568972kB, anon-rss:3316388kB, file-rss:32kB
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261526] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261530] php cpuset=/ mems_allowed=0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261533] Pid: 23428, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261534] Call Trace:
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261542] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261544] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261546] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261549] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261552] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261557] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261560] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261562] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261565] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261568] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261571] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261573] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261575] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261578] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261582] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261584] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261587] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261590] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261593] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261595] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261598] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261599] Mem-Info:
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261601] Node 0 DMA per-cpu:
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261603] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261604] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261605] Node 0 DMA32 per-cpu:
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261607] CPU 0: hi: 186, btch: 31 usd: 88
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261608] CPU 1: hi: 186, btch: 31 usd: 83
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261611] active_anon:932562 inactive_anon:1393 isolated_anon:0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261612] active_file:11 inactive_file:3 isolated_file:23
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261612] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261613] free:4031 slab_reclaimable:2606 slab_unreclaimable:2570
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261614] mapped:1378 shmem:1405 pagetables:3325 bounce:0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261615] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261622] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261625] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3730248kB inactive_anon:5572kB active_file:44kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):92kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5512kB shmem:5620kB slab_reclaimable:10424kB slab_unreclaimable:10280kB kernel_stack:792kB pagetables:13300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:385 all_unreclaimable? yes
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261631] lowmem_reserve[]: 0 0 0 0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261634] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261640] Node 0 DMA32: 1043*4kB 178*8kB 7*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7788kB
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261647] 1454 total pagecache pages
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261649] 0 pages in swap cache
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261650] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261651] Free swap = 0kB
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.261652] Total swap = 0kB
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266875] 985072 pages RAM
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266877] 24998 pages reserved
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266878] 3777 pages shared
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266880] 953198 pages non-shared
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266881] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266890] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266894] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266897] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266900] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266904] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266907] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266910] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266914] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266918] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266921] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266924] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266927] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266930] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266934] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266937] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266940] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266943] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266946] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266950] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266953] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266956] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266959] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266962] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266965] [13468] 0 13468 89584 2516 0 0 0 php5-fpm
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266968] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266971] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266975] [ 2686] 1012 2686 68618 5664 1 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266978] [16697] 1012 16697 879891 816800 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266981] [18173] 108 18173 6792 69 1 0 0 pickup
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266984] [23428] 1012 23428 68580 5604 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266987] [23490] 1012 23490 1099 25 0 0 0 sh
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266990] [23491] 1012 23491 68825 5873 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266993] [23497] 1012 23497 1099 26 0 0 0 sh
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266996] [23498] 1012 23498 4655 65 0 0 0 git
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.266999] [23499] 1012 23499 68499 5551 0 0 0 php
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.267002] [23501] 1012 23501 1099 24 0 0 0 sh
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.267006] [23502] 1012 23502 10901 165 0 0 0 ssh
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.267009] [23504] 1012 23504 4647 40 1 0 0 git
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.267011] Out of memory: Kill process 16697 (php) score 852 or sacrifice child
Jul 15 15:52:29 ip-10-61-174-46 kernel: [1430445.267022] Killed process 16697 (php) total-vm:3519564kB, anon-rss:3267200kB, file-rss:0kB
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680204] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680208] php cpuset=/ mems_allowed=0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680211] Pid: 23523, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680213] Call Trace:
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680220] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680222] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680224] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680228] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680232] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680235] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680238] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680241] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680243] [<ffffffff8111a6c1>] ? filemap_fault+0x1/0x3c0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680246] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680249] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680251] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680255] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680257] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680259] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680262] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680265] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680267] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680269] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680271] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680273] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680276] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680278] Mem-Info:
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680279] Node 0 DMA per-cpu:
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680281] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680282] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680283] Node 0 DMA32 per-cpu:
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680285] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680286] CPU 1: hi: 186, btch: 31 usd: 2
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680288] active_anon:932870 inactive_anon:1393 isolated_anon:0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680289] active_file:0 inactive_file:0 isolated_file:0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680290] unevictable:0 dirty:19 writeback:0 unstable:0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680290] free:4048 slab_reclaimable:2547 slab_unreclaimable:2584
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680291] mapped:1356 shmem:1405 pagetables:3192 bounce:0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680293] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680299] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680302] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731480kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:76kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10188kB slab_unreclaimable:10336kB kernel_stack:768kB pagetables:12768kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4 all_unreclaimable? yes
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680309] lowmem_reserve[]: 0 0 0 0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680311] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680318] Node 0 DMA32: 862*4kB 230*8kB 21*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7864kB
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680324] 1462 total pagecache pages
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680326] 0 pages in swap cache
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680327] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680328] Free swap = 0kB
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.680329] Total swap = 0kB
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685675] 985072 pages RAM
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685678] 24998 pages reserved
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685679] 3788 pages shared
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685680] 953310 pages non-shared
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685682] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685691] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685695] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685698] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685701] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685704] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685708] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685711] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685714] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685718] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685721] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685725] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685728] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685731] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685734] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685737] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685740] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685744] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685747] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685750] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685753] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685756] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685759] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685762] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685765] [13468] 0 13468 89584 2501 1 0 0 php5-fpm
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685768] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685772] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685775] [ 2686] 1012 2686 68618 5663 1 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685778] [18173] 108 18173 6792 75 1 0 0 pickup
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685781] [23523] 1012 23523 891326 828244 0 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685784] [24233] 108 24233 6818 73 1 0 0 cleanup
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685788] [24234] 108 24234 6795 70 0 0 0 trivial-rewrite
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685791] [24235] 108 24235 11107 158 0 0 0 smtp
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685794] [24239] 108 24239 11107 159 1 0 0 smtp
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685797] [24243] 108 24243 11107 157 1 0 0 smtp
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685800] [24627] 1012 24627 68580 5604 1 0 0 php
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685803] Out of memory: Kill process 23523 (php) score 864 or sacrifice child
Jul 15 15:55:42 ip-10-61-174-46 kernel: [1430637.685814] Killed process 23523 (php) total-vm:3565304kB, anon-rss:3312976kB, file-rss:0kB
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620121] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620126] php cpuset=/ mems_allowed=0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620129] Pid: 24771, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620131] Call Trace:
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620142] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620145] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620148] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620152] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620154] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620157] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620160] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620162] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620165] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620167] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620171] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620173] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620175] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620178] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620180] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620182] Mem-Info:
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620183] Node 0 DMA per-cpu:
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620185] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620187] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620188] Node 0 DMA32 per-cpu:
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620189] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620191] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620193] active_anon:932942 inactive_anon:1393 isolated_anon:0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620194] active_file:35 inactive_file:40 isolated_file:0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620195] unevictable:0 dirty:22 writeback:0 unstable:0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620195] free:4048 slab_reclaimable:2715 slab_unreclaimable:2553
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620196] mapped:1353 shmem:1405 pagetables:3062 bounce:0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620197] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620204] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620207] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731768kB inactive_anon:5572kB active_file:140kB inactive_file:160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10860kB slab_unreclaimable:10212kB kernel_stack:728kB pagetables:12248kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24947 all_unreclaimable? yes
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620213] lowmem_reserve[]: 0 0 0 0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620216] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620223] Node 0 DMA32: 923*4kB 191*8kB 27*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7892kB
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620229] 1447 total pagecache pages
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620231] 0 pages in swap cache
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620232] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620233] Free swap = 0kB
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.620234] Total swap = 0kB
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625451] 985072 pages RAM
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625454] 24998 pages reserved
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625455] 3801 pages shared
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625456] 953327 pages non-shared
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625458] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625466] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625470] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625474] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625477] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625480] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625483] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625486] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625489] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625494] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625497] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625500] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625503] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625506] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625509] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625512] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625516] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625519] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625522] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625525] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625528] [ 654] 1012 654 70351 5760 0 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625531] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625534] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625537] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625540] [13468] 0 13468 89584 2540 0 0 0 php5-fpm
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625543] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625546] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625549] [ 2686] 1012 2686 68618 5664 1 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625553] [18173] 108 18173 6792 75 1 0 0 pickup
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625556] [24771] 1012 24771 892039 828971 0 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625559] [ 1525] 1012 1525 68580 5605 0 0 0 php
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625562] Out of memory: Kill process 24771 (php) score 865 or sacrifice child
Jul 15 16:21:57 ip-10-61-174-46 kernel: [1432212.625573] Killed process 24771 (php) total-vm:3568156kB, anon-rss:3315788kB, file-rss:96kB
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052115] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052119] php cpuset=/ mems_allowed=0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052122] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052123] Call Trace:
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052130] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052132] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052134] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052137] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052140] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052144] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052146] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052149] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052151] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052155] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052157] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052160] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052162] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052165] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052168] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052170] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052172] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052175] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052176] Mem-Info:
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052178] Node 0 DMA per-cpu:
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052179] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052181] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052182] Node 0 DMA32 per-cpu:
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052184] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052185] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052187] active_anon:933153 inactive_anon:1393 isolated_anon:0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052188] active_file:0 inactive_file:0 isolated_file:0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052189] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052189] free:4048 slab_reclaimable:2490 slab_unreclaimable:2555
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052190] mapped:1379 shmem:1405 pagetables:3079 bounce:0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052192] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052198] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052201] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732612kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5620kB slab_reclaimable:9960kB slab_unreclaimable:10220kB kernel_stack:728kB pagetables:12316kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052208] lowmem_reserve[]: 0 0 0 0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052210] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052217] Node 0 DMA32: 942*4kB 233*8kB 13*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052224] 1461 total pagecache pages
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052225] 0 pages in swap cache
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052226] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052227] Free swap = 0kB
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.052228] Total swap = 0kB
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057517] 985072 pages RAM
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057520] 24998 pages reserved
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057521] 3795 pages shared
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057522] 953324 pages non-shared
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057524] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057533] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057536] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057539] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057542] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057545] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057549] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057552] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057555] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057559] [ 790] 101 790 63477 632 1 0 0 rsyslogd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057562] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057565] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057568] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057571] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057574] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057578] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057581] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057584] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057587] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057590] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057593] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057596] [ 657] 1012 657 69242 6261 0 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057599] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057602] [ 678] 1012 678 70152 5571 1 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057605] [13468] 0 13468 89584 2508 0 0 0 php5-fpm
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057608] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057611] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057614] [ 2686] 1012 2686 68618 5663 1 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057618] [18173] 108 18173 6792 75 0 0 0 pickup
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057621] [ 1559] 1012 1559 892271 829128 0 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057625] [ 4222] 1012 4222 68580 5604 1 0 0 php
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057627] Out of memory: Kill process 1559 (php) score 865 or sacrifice child
Jul 15 16:29:52 ip-10-61-174-46 kernel: [1432688.057638] Killed process 1559 (php) total-vm:3569084kB, anon-rss:3316512kB, file-rss:0kB
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956319] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956323] php cpuset=/ mems_allowed=0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956325] Pid: 4360, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956327] Call Trace:
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956334] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956336] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956339] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956341] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956346] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956349] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956352] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956354] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956357] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956360] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956362] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956366] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956368] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956370] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956373] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956375] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956378] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956380] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956383] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956386] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956387] Mem-Info:
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956388] Node 0 DMA per-cpu:
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956390] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956392] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956393] Node 0 DMA32 per-cpu:
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956394] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956396] CPU 1: hi: 186, btch: 31 usd: 28
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956398] active_anon:933303 inactive_anon:1393 isolated_anon:0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956399] active_file:0 inactive_file:2 isolated_file:0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956400] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956400] free:4032 slab_reclaimable:2339 slab_unreclaimable:2562
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956401] mapped:1356 shmem:1405 pagetables:3073 bounce:0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956402] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956409] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956412] Node 0 DMA32 free:7792kB min:7848kB low:9808kB high:11772kB active_anon:3733212kB inactive_anon:5572kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:9356kB slab_unreclaimable:10248kB kernel_stack:736kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:33 all_unreclaimable? yes
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956418] lowmem_reserve[]: 0 0 0 0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956421] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956428] Node 0 DMA32: 586*4kB 161*8kB 96*16kB 18*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7792kB
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956434] 1435 total pagecache pages
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956436] 0 pages in swap cache
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956437] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956438] Free swap = 0kB
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.956439] Total swap = 0kB
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961626] 985072 pages RAM
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961628] 24998 pages reserved
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961630] 3781 pages shared
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961631] 953313 pages non-shared
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961633] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961642] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961645] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961648] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961651] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961655] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961658] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961661] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961665] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961669] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961672] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961675] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961678] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961681] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961684] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961687] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961690] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961694] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961697] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961700] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961703] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961706] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961709] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961712] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961715] [13468] 0 13468 89584 2493 1 0 0 php5-fpm
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961718] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961721] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961724] [ 2686] 1012 2686 68618 5663 0 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961728] [18173] 108 18173 6792 75 0 0 0 pickup
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961731] [ 4360] 1012 4360 892435 829289 0 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961734] [ 4476] 1012 4476 68580 5605 1 0 0 php
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961737] Out of memory: Kill process 4360 (php) score 865 or sacrifice child
Jul 15 16:30:35 ip-10-61-174-46 kernel: [1432730.961748] Killed process 4360 (php) total-vm:3569740kB, anon-rss:3317148kB, file-rss:8kB
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144123] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144127] php cpuset=/ mems_allowed=0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144130] Pid: 5399, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144131] Call Trace:
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144141] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144143] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144146] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144151] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144154] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144158] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144160] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144163] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144166] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144168] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144171] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144173] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144176] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144178] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144181] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144184] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144187] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144191] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144194] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144197] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144200] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144201] Mem-Info:
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144202] Node 0 DMA per-cpu:
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144204] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144206] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144207] Node 0 DMA32 per-cpu:
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144208] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144210] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144212] active_anon:933207 inactive_anon:1393 isolated_anon:0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144213] active_file:0 inactive_file:0 isolated_file:0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144213] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144214] free:4048 slab_reclaimable:2428 slab_unreclaimable:2529
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144215] mapped:1355 shmem:1405 pagetables:3073 bounce:0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144216] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144223] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732828kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:9712kB slab_unreclaimable:10116kB kernel_stack:736kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144232] lowmem_reserve[]: 0 0 0 0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144235] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144241] Node 0 DMA32: 678*4kB 161*8kB 87*16kB 11*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144248] 1475 total pagecache pages
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144250] 0 pages in swap cache
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144251] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144252] Free swap = 0kB
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.144253] Total swap = 0kB
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149498] 985072 pages RAM
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149500] 24998 pages reserved
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149501] 3804 pages shared
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149503] 953333 pages non-shared
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149505] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149513] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149517] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149520] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149523] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149526] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149530] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149533] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149536] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149540] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149543] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149547] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149550] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149553] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149556] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149559] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149562] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149565] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149569] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149572] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149575] [ 654] 1012 654 70351 5760 1 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149578] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149581] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149585] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149588] [13468] 0 13468 89584 2505 0 0 0 php5-fpm
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149591] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149594] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149597] [ 2686] 1012 2686 68618 5663 1 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149601] [18173] 108 18173 6792 75 0 0 0 pickup
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149604] [ 4655] 1012 4655 892307 829183 1 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149607] [ 5399] 1012 5399 68580 5603 1 0 0 php
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149610] Out of memory: Kill process 4655 (php) score 865 or sacrifice child
Jul 15 16:32:48 ip-10-61-174-46 kernel: [1432864.149621] Killed process 4655 (php) total-vm:3569228kB, anon-rss:3316732kB, file-rss:0kB
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444833] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444837] php cpuset=/ mems_allowed=0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444840] Pid: 5460, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444841] Call Trace:
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444848] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444851] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444853] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444856] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444861] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444864] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444867] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444870] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444872] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444875] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444878] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444880] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444883] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444886] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444888] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444891] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444893] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444895] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444898] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444901] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444903] Mem-Info:
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444904] Node 0 DMA per-cpu:
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444906] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444907] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444908] Node 0 DMA32 per-cpu:
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444910] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444911] CPU 1: hi: 186, btch: 31 usd: 27
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444914] active_anon:933188 inactive_anon:1393 isolated_anon:0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444914] active_file:5 inactive_file:10 isolated_file:0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444915] unevictable:0 dirty:11 writeback:0 unstable:0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444916] free:4036 slab_reclaimable:2475 slab_unreclaimable:2536
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444917] mapped:1369 shmem:1405 pagetables:3061 bounce:0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444918] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444925] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444928] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3732752kB inactive_anon:5572kB active_file:20kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:44kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:9900kB slab_unreclaimable:10144kB kernel_stack:728kB pagetables:12244kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:118 all_unreclaimable? yes
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444934] lowmem_reserve[]: 0 0 0 0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444936] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444943] Node 0 DMA32: 720*4kB 166*8kB 37*16kB 26*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444950] 1491 total pagecache pages
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444951] 0 pages in swap cache
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444952] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444953] Free swap = 0kB
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.444954] Total swap = 0kB
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450335] 985072 pages RAM
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450338] 24998 pages reserved
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450339] 3815 pages shared
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450340] 953277 pages non-shared
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450342] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450351] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450354] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450357] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450361] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450364] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450367] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450370] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450374] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450378] [ 790] 101 790 63477 632 1 0 0 rsyslogd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450381] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450384] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450387] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450390] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450393] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450397] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450400] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450403] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450406] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450409] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450412] [ 654] 1012 654 70351 5761 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450415] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450418] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450422] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450425] [13468] 0 13468 89584 2538 0 0 0 php5-fpm
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450428] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450431] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450434] [ 2686] 1012 2686 68618 5664 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450437] [18173] 108 18173 6792 75 1 0 0 pickup
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450440] [ 5460] 1012 5460 892271 829204 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450444] [10653] 1012 10653 68580 5605 0 0 0 php
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450446] Out of memory: Kill process 5460 (php) score 865 or sacrifice child
Jul 15 16:48:24 ip-10-61-174-46 kernel: [1433800.450458] Killed process 5460 (php) total-vm:3569084kB, anon-rss:3316744kB, file-rss:72kB
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328098] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328102] php cpuset=/ mems_allowed=0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328105] Pid: 10685, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328106] Call Trace:
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328114] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328116] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328118] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328121] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328125] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328128] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328131] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328134] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328136] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328139] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328142] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328145] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328148] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328150] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328152] Mem-Info:
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328153] Node 0 DMA per-cpu:
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328155] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328156] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328157] Node 0 DMA32 per-cpu:
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328159] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328160] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328163] active_anon:933273 inactive_anon:1393 isolated_anon:0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328164] active_file:2 inactive_file:0 isolated_file:0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328164] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328165] free:4048 slab_reclaimable:2374 slab_unreclaimable:2584
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328166] mapped:1355 shmem:1405 pagetables:3079 bounce:0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328167] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328174] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328177] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733092kB inactive_anon:5572kB active_file:8kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:9496kB slab_unreclaimable:10336kB kernel_stack:728kB pagetables:12316kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:16 all_unreclaimable? yes
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328183] lowmem_reserve[]: 0 0 0 0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328186] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328193] Node 0 DMA32: 326*4kB 113*8kB 124*16kB 42*32kB 5*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8032kB
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328200] 1427 total pagecache pages
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328201] 0 pages in swap cache
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328202] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328203] Free swap = 0kB
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.328204] Total swap = 0kB
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333599] 985072 pages RAM
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333602] 24998 pages reserved
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333603] 3775 pages shared
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333604] 953318 pages non-shared
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333606] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333614] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333618] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333621] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333624] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333628] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333631] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333634] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333638] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333642] [ 790] 101 790 63477 632 1 0 0 rsyslogd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333645] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333649] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333652] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333655] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333658] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333661] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333664] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333667] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333670] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333674] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333677] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333680] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333683] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333686] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333689] [13468] 0 13468 89584 2523 1 0 0 php5-fpm
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333692] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333695] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333698] [ 2686] 1012 2686 68618 5664 0 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333702] [18173] 108 18173 6792 75 0 0 0 pickup
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333705] [10685] 1012 10685 892371 829223 0 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333708] [10731] 1012 10731 68580 5604 1 0 0 php
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333711] Out of memory: Kill process 10685 (php) score 865 or sacrifice child
Jul 15 16:49:01 ip-10-61-174-46 kernel: [1433837.333722] Killed process 10685 (php) total-vm:3569484kB, anon-rss:3316888kB, file-rss:4kB
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612160] sh invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612164] sh cpuset=/ mems_allowed=0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612167] Pid: 30803, comm: sh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612170] Call Trace:
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612178] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612181] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612184] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612187] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612192] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612197] [<ffffffff811894c3>] ? user_path_at_empty+0x63/0xa0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612200] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612205] [<ffffffff81196e8e>] ? vfsmount_lock_local_unlock+0x1e/0x30
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612208] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612211] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612214] [<ffffffff8117e6aa>] ? sys_newstat+0x2a/0x40
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612218] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612223] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612226] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612228] Mem-Info:
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612230] Node 0 DMA per-cpu:
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612232] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612234] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612236] Node 0 DMA32 per-cpu:
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612238] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612240] CPU 1: hi: 186, btch: 31 usd: 14
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612243] active_anon:932431 inactive_anon:1393 isolated_anon:0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612244] active_file:13 inactive_file:18 isolated_file:0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612245] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612246] free:4048 slab_reclaimable:2812 slab_unreclaimable:2622
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612247] mapped:1353 shmem:1405 pagetables:3303 bounce:0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612249] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612271] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612275] Node 0 DMA32 free:7864kB min:7848kB low:9808kB high:11772kB active_anon:3729724kB inactive_anon:5572kB active_file:52kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:11248kB slab_unreclaimable:10488kB kernel_stack:800kB pagetables:13212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612283] lowmem_reserve[]: 0 0 0 0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612287] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612306] Node 0 DMA32: 1417*4kB 22*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7924kB
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612315] 1466 total pagecache pages
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612316] 0 pages in swap cache
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612324] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612333] Free swap = 0kB
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.612334] Total swap = 0kB
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619433] 985072 pages RAM
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619436] 24998 pages reserved
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619437] 3791 pages shared
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619438] 953291 pages non-shared
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619440] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619448] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619452] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619455] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619457] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619461] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619464] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619467] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619470] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619474] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619477] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619480] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619483] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619486] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619489] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619492] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619495] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619498] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619500] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619503] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619506] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619509] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619512] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619515] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619518] [13468] 0 13468 89584 2534 1 0 0 php5-fpm
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619521] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619524] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619527] [ 2686] 1012 2686 68618 5666 1 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619531] [16949] 108 16949 6792 70 1 0 0 pickup
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619534] [26393] 1012 26393 885010 821718 0 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619537] [29953] 108 29953 6818 73 1 0 0 cleanup
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619541] [29954] 108 29954 6795 71 0 0 0 trivial-rewrite
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619544] [29955] 108 29955 11107 157 0 0 0 smtp
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619547] [30409] 108 30409 11107 159 0 0 0 smtp
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619550] [30413] 108 30413 11107 158 1 0 0 smtp
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619553] [30417] 108 30417 11107 159 1 0 0 smtp
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619556] [30673] 1012 30673 68580 5603 1 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619559] [30796] 1012 30796 1099 24 0 0 0 sh
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619563] [30797] 1012 30797 68825 5870 0 0 0 php
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619566] [30803] 1012 30803 1099 26 0 0 0 sh
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619568] Out of memory: Kill process 26393 (php) score 857 or sacrifice child
Jul 15 17:43:09 ip-10-61-174-46 kernel: [1437084.619579] Killed process 26393 (php) total-vm:3540040kB, anon-rss:3286832kB, file-rss:40kB
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086307] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086311] php cpuset=/ mems_allowed=0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086314] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086316] Call Trace:
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086323] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086325] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086328] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086331] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086336] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086338] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086342] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086344] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086347] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086349] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086353] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086355] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086357] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086361] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086364] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086367] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086372] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086375] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086377] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086380] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086382] Mem-Info:
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086383] Node 0 DMA per-cpu:
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086385] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086386] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086387] Node 0 DMA32 per-cpu:
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086389] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086390] CPU 1: hi: 186, btch: 31 usd: 46
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086393] active_anon:932641 inactive_anon:1393 isolated_anon:0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086394] active_file:1 inactive_file:0 isolated_file:0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086394] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086395] free:4046 slab_reclaimable:2841 slab_unreclaimable:2597
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086396] mapped:1361 shmem:1405 pagetables:3135 bounce:0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086397] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086404] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086407] Node 0 DMA32 free:7848kB min:7848kB low:9808kB high:11772kB active_anon:3730564kB inactive_anon:5572kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:11364kB slab_unreclaimable:10388kB kernel_stack:752kB pagetables:12540kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:229 all_unreclaimable? yes
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086413] lowmem_reserve[]: 0 0 0 0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086416] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086422] Node 0 DMA32: 1452*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086429] 1431 total pagecache pages
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086430] 0 pages in swap cache
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086431] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086432] Free swap = 0kB
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.086433] Total swap = 0kB
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091765] 985072 pages RAM
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091767] 24998 pages reserved
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091768] 3781 pages shared
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091769] 953277 pages non-shared
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091771] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091780] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091783] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091786] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091790] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091793] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091796] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091799] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091803] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091807] [ 790] 101 790 63477 632 0 0 0 rsyslogd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091810] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091813] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091816] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091820] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091823] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091826] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091829] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091832] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091835] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091839] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091842] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091845] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091848] [ 658] 1012 658 134457 70862 1 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091851] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091854] [13468] 0 13468 89584 2501 0 0 0 php5-fpm
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091857] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091860] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091863] [ 2686] 1012 2686 68618 5667 1 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091867] [ 515] 108 515 6792 69 0 0 0 pickup
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091870] [ 5491] 1012 5491 891640 828298 1 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091874] [ 6500] 108 6500 6818 74 0 0 0 cleanup
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091877] [ 6501] 108 6501 6795 70 0 0 0 trivial-rewrite
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091880] [ 6502] 108 6502 11107 158 0 0 0 smtp
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091883] [ 7356] 1012 7356 68580 5604 1 0 0 php
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091886] Out of memory: Kill process 5491 (php) score 864 or sacrifice child
Jul 15 18:06:45 ip-10-61-174-46 kernel: [1438501.091897] Killed process 5491 (php) total-vm:3566560kB, anon-rss:3313192kB, file-rss:0kB
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268216] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268221] php cpuset=/ mems_allowed=0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268225] Pid: 12185, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268227] Call Trace:
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268235] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268238] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268241] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268245] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268249] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268254] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268259] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268262] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268265] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268269] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268273] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268276] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268279] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268282] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268286] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268289] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268292] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268296] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268299] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268303] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268306] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268308] Mem-Info:
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268310] Node 0 DMA per-cpu:
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268312] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268314] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268316] Node 0 DMA32 per-cpu:
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268318] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268319] CPU 1: hi: 186, btch: 31 usd: 27
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268323] active_anon:933098 inactive_anon:1393 isolated_anon:0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268324] active_file:0 inactive_file:8 isolated_file:0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268324] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268325] free:4018 slab_reclaimable:2586 slab_unreclaimable:2581
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268326] mapped:1362 shmem:1405 pagetables:3078 bounce:0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268328] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268337] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268341] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3732392kB inactive_anon:5572kB active_file:0kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:10344kB slab_unreclaimable:10324kB kernel_stack:728kB pagetables:12312kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:34 all_unreclaimable? yes
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268349] lowmem_reserve[]: 0 0 0 0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268353] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268362] Node 0 DMA32: 1238*4kB 92*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268371] 1409 total pagecache pages
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268373] 0 pages in swap cache
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268374] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268376] Free swap = 0kB
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.268377] Total swap = 0kB
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273603] 985072 pages RAM
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273606] 24998 pages reserved
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273607] 3782 pages shared
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273609] 953314 pages non-shared
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273611] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273620] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273623] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273626] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273629] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273632] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273636] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273639] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273642] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273646] [ 790] 101 790 63477 634 1 0 0 rsyslogd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273649] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273652] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273656] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273659] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273662] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273665] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273668] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273671] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273674] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273678] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273681] [ 654] 1012 654 70351 5762 1 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273684] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273687] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273690] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273693] [13468] 0 13468 89584 2529 1 0 0 php5-fpm
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273696] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273699] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273702] [ 2686] 1012 2686 68618 5668 1 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273706] [ 515] 108 515 6792 75 0 0 0 pickup
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273709] [ 7480] 1012 7480 892238 829026 0 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273712] [12185] 1012 12185 68580 5605 1 0 0 php
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273715] Out of memory: Kill process 7480 (php) score 865 or sacrifice child
Jul 15 18:19:07 ip-10-61-174-46 kernel: [1439243.273725] Killed process 7480 (php) total-vm:3568952kB, anon-rss:3316100kB, file-rss:4kB
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804908] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804912] php cpuset=/ mems_allowed=0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804915] Pid: 12652, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804916] Call Trace:
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804924] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804926] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804928] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804931] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804934] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804938] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804940] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804943] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804945] [<ffffffff8113c9e8>] ? do_wp_page+0x2e8/0x740
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804949] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804951] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804953] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804956] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804958] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804962] [<ffffffff8105761b>] ? check_preempt_wakeup+0x15b/0x230
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804966] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804969] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804973] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804975] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804978] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804980] Mem-Info:
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804981] Node 0 DMA per-cpu:
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804983] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804984] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804985] Node 0 DMA32 per-cpu:
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804988] CPU 0: hi: 186, btch: 31 usd: 2
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804990] CPU 1: hi: 186, btch: 31 usd: 36
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804993] active_anon:933034 inactive_anon:1393 isolated_anon:0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804993] active_file:21 inactive_file:29 isolated_file:0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804994] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804995] free:4050 slab_reclaimable:2324 slab_unreclaimable:2581
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804996] mapped:1350 shmem:1405 pagetables:3282 bounce:0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.804998] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805006] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805010] Node 0 DMA32 free:7864kB min:7848kB low:9808kB high:11772kB active_anon:3732136kB inactive_anon:5572kB active_file:84kB inactive_file:116kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:9296kB slab_unreclaimable:10324kB kernel_stack:768kB pagetables:13128kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:337 all_unreclaimable? yes
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805019] lowmem_reserve[]: 0 0 0 0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805022] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805036] Node 0 DMA32: 374*4kB 47*8kB 108*16kB 49*32kB 5*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7664kB
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805045] 1496 total pagecache pages
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805047] 0 pages in swap cache
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805048] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805050] Free swap = 0kB
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.805051] Total swap = 0kB
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810862] 985072 pages RAM
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810864] 24998 pages reserved
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810866] 3785 pages shared
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810867] 953295 pages non-shared
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810869] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810878] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810881] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810884] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810888] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810891] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810895] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810898] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810901] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810905] [ 790] 101 790 63477 634 1 0 0 rsyslogd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810908] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810911] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810915] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810918] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810921] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810924] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810927] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810930] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810934] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810937] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810940] [ 654] 1012 654 70351 5761 0 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810943] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810946] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810949] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810952] [13468] 0 13468 89584 2496 1 0 0 php5-fpm
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810955] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810958] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810962] [ 2686] 1012 2686 68618 5667 1 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810965] [ 515] 108 515 6792 75 0 0 0 pickup
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810969] [12279] 1012 12279 884668 821508 1 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810971] [12558] 1012 12558 68580 5603 1 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810974] [12651] 1012 12651 1099 24 1 0 0 sh
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810977] [12652] 1012 12652 68825 5870 1 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810980] [12658] 1012 12658 1099 26 1 0 0 sh
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810983] [12659] 1012 12659 4651 57 1 0 0 git
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810986] [12660] 1012 12660 64155 1517 0 0 0 php
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.810989] Out of memory: Kill process 12279 (php) score 857 or sacrifice child
Jul 15 18:20:02 ip-10-61-174-46 kernel: [1439297.811001] Killed process 12279 (php) total-vm:3538672kB, anon-rss:3286032kB, file-rss:0kB
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048120] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048124] php cpuset=/ mems_allowed=0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048127] Pid: 12752, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048129] Call Trace:
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048136] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048138] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048140] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048143] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048146] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048150] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048152] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048155] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048158] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048160] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048163] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048165] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048169] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048171] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048174] [<ffffffff811425f9>] ? find_vma+0x49/0x70
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048177] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048178] Mem-Info:
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048179] Node 0 DMA per-cpu:
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048181] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048183] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048184] Node 0 DMA32 per-cpu:
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048185] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048187] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048189] active_anon:933167 inactive_anon:1393 isolated_anon:0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048190] active_file:22 inactive_file:37 isolated_file:0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048190] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048191] free:4048 slab_reclaimable:2507 slab_unreclaimable:2574
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048192] mapped:1380 shmem:1405 pagetables:3074 bounce:0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048193] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048200] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048203] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732668kB inactive_anon:5572kB active_file:88kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5520kB shmem:5620kB slab_reclaimable:10028kB slab_unreclaimable:10296kB kernel_stack:728kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:31246 all_unreclaimable? yes
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048209] lowmem_reserve[]: 0 0 0 0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048212] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048219] Node 0 DMA32: 618*4kB 177*8kB 62*16kB 25*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048225] 1449 total pagecache pages
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048226] 0 pages in swap cache
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048228] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048229] Free swap = 0kB
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.048230] Total swap = 0kB
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053453] 985072 pages RAM
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053455] 24998 pages reserved
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053456] 3793 pages shared
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053457] 953344 pages non-shared
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053459] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053468] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053472] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053475] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053478] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053481] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053484] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053488] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053491] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053495] [ 790] 101 790 63477 634 1 0 0 rsyslogd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053498] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053501] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053505] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053507] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053511] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053514] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053517] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053520] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053523] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053526] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053530] [ 654] 1012 654 70351 5762 1 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053533] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053536] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053539] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053542] [13468] 0 13468 89584 2526 1 0 0 php5-fpm
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053545] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053548] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053551] [ 2686] 1012 2686 68618 5667 1 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053555] [ 515] 108 515 6792 75 0 0 0 pickup
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053558] [12752] 1012 12752 892285 829137 0 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053561] [13374] 1012 13374 68580 5605 1 0 0 php
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053564] Out of memory: Kill process 12752 (php) score 865 or sacrifice child
Jul 15 18:21:59 ip-10-61-174-46 kernel: [1439415.053575] Killed process 12752 (php) total-vm:3569140kB, anon-rss:3316508kB, file-rss:40kB
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824814] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824817] php cpuset=/ mems_allowed=0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824820] Pid: 13553, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824822] Call Trace:
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824829] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824831] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824834] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824836] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824840] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824843] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824846] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824849] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824851] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824854] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824857] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824859] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824862] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824864] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824868] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824870] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824872] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824875] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824877] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824879] Mem-Info:
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824880] Node 0 DMA per-cpu:
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824882] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824884] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824885] Node 0 DMA32 per-cpu:
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824886] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824888] CPU 1: hi: 186, btch: 31 usd: 30
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824890] active_anon:933255 inactive_anon:1393 isolated_anon:0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824891] active_file:1 inactive_file:0 isolated_file:0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824892] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824892] free:4027 slab_reclaimable:2361 slab_unreclaimable:2540
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824893] mapped:1353 shmem:1405 pagetables:3076 bounce:0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824894] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824901] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824904] Node 0 DMA32 free:7772kB min:7848kB low:9808kB high:11772kB active_anon:3733020kB inactive_anon:5572kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:9444kB slab_unreclaimable:10160kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:130 all_unreclaimable? yes
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824910] lowmem_reserve[]: 0 0 0 0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824913] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824919] Node 0 DMA32: 591*4kB 140*8kB 102*16kB 19*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7772kB
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824926] 1419 total pagecache pages
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824927] 0 pages in swap cache
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824928] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824929] Free swap = 0kB
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.824930] Total swap = 0kB
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830160] 985072 pages RAM
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830163] 24998 pages reserved
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830164] 3778 pages shared
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830165] 953320 pages non-shared
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830167] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830176] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830179] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830182] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830185] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830189] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830192] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830196] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830199] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830203] [ 790] 101 790 63477 634 1 0 0 rsyslogd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830206] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830209] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830213] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830216] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830219] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830222] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830225] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830228] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830231] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830235] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830238] [ 654] 1012 654 70351 5761 1 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830241] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830244] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830247] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830250] [13468] 0 13468 89584 2511 1 0 0 php5-fpm
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830253] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830256] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830259] [ 2686] 1012 2686 68618 5667 1 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830263] [ 515] 108 515 6792 75 1 0 0 pickup
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830266] [13553] 1012 13553 892349 829224 0 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830269] [13709] 1012 13709 68580 5603 1 0 0 php
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830272] Out of memory: Kill process 13553 (php) score 865 or sacrifice child
Jul 15 18:22:54 ip-10-61-174-46 kernel: [1439469.830283] Killed process 13553 (php) total-vm:3569396kB, anon-rss:3316896kB, file-rss:0kB
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903587] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903592] php cpuset=/ mems_allowed=0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903595] Pid: 15884, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903596] Call Trace:
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903603] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903606] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903608] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903611] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903616] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903619] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903622] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903625] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903628] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903632] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903634] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903636] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903639] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903642] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903644] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903647] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903649] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903652] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903654] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903657] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903661] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903664] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903667] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903669] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903671] Mem-Info:
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903672] Node 0 DMA per-cpu:
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903674] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903675] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903676] Node 0 DMA32 per-cpu:
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903678] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903679] CPU 1: hi: 186, btch: 31 usd: 17
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903682] active_anon:933145 inactive_anon:1393 isolated_anon:0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903682] active_file:3 inactive_file:0 isolated_file:0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903683] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903684] free:4048 slab_reclaimable:2486 slab_unreclaimable:2561
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903684] mapped:1366 shmem:1405 pagetables:3093 bounce:0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903686] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903693] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903696] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732580kB inactive_anon:5572kB active_file:12kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9944kB slab_unreclaimable:10244kB kernel_stack:736kB pagetables:12372kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:103 all_unreclaimable? yes
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903702] lowmem_reserve[]: 0 0 0 0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903704] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903711] Node 0 DMA32: 805*4kB 161*8kB 46*16kB 18*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903718] 1426 total pagecache pages
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903719] 0 pages in swap cache
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903720] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903721] Free swap = 0kB
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.903722] Total swap = 0kB
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.908994] 985072 pages RAM
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.908996] 24998 pages reserved
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.908997] 3788 pages shared
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.908998] 953313 pages non-shared
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909000] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909009] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909013] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909016] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909019] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909023] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909027] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909030] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909034] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909038] [ 790] 101 790 63477 634 1 0 0 rsyslogd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909041] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909044] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909047] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909050] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909053] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909056] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909060] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909062] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909066] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909069] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909072] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909075] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909078] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909081] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909084] [13468] 0 13468 89584 2496 0 0 0 php5-fpm
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909087] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909090] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909093] [ 2686] 1012 2686 68618 5667 0 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909097] [ 515] 108 515 6792 75 1 0 0 pickup
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909100] [13803] 1012 13803 892142 829043 1 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909103] [15114] 108 15114 6792 69 1 0 0 scache
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909106] [15884] 1012 15884 68580 5604 0 0 0 php
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909109] Out of memory: Kill process 13803 (php) score 865 or sacrifice child
Jul 15 18:28:21 ip-10-61-174-46 kernel: [1439796.909120] Killed process 13803 (php) total-vm:3568568kB, anon-rss:3316172kB, file-rss:0kB
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904122] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904127] php5-fpm cpuset=/ mems_allowed=0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904130] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904131] Call Trace:
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904141] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904143] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904146] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904149] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904155] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904158] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904162] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904164] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904172] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904175] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904177] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904180] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904183] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904187] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904192] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904195] [<ffffffff8117ae2d>] ? fget+0x1d/0x90
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904197] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904199] Mem-Info:
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904200] Node 0 DMA per-cpu:
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904202] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904203] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904204] Node 0 DMA32 per-cpu:
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904206] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904207] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904210] active_anon:932364 inactive_anon:1393 isolated_anon:0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904210] active_file:0 inactive_file:0 isolated_file:0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904211] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904212] free:4048 slab_reclaimable:3023 slab_unreclaimable:2637
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904212] mapped:1358 shmem:1405 pagetables:3212 bounce:0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904214] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904221] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904224] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729456kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:12092kB slab_unreclaimable:10548kB kernel_stack:776kB pagetables:12848kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904230] lowmem_reserve[]: 0 0 0 0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904233] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904239] Node 0 DMA32: 1359*4kB 48*8kB 3*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904246] 1476 total pagecache pages
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904247] 0 pages in swap cache
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904249] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904250] Free swap = 0kB
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.904250] Total swap = 0kB
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909389] 985072 pages RAM
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909392] 24998 pages reserved
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909393] 3784 pages shared
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909394] 953324 pages non-shared
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909396] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909404] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909408] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909411] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909414] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909417] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909421] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909424] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909427] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909431] [ 790] 101 790 63477 634 1 0 0 rsyslogd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909434] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909438] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909441] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909444] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909447] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909450] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909453] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909456] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909459] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909462] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909465] [ 654] 1012 654 70351 5762 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909469] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909471] [ 658] 1012 658 134457 70863 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909475] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909478] [13468] 0 13468 89584 2530 0 0 0 php5-fpm
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909481] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909484] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909487] [ 2686] 1012 2686 68618 5667 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909491] [ 515] 108 515 6792 75 1 0 0 pickup
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909494] [16007] 1012 16007 885390 822200 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909497] [17680] 1012 17680 73917 10949 1 0 0 php
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909500] [17694] 108 17694 6818 74 1 0 0 cleanup
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909503] [17695] 108 17695 6795 71 1 0 0 trivial-rewrite
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909506] [17698] 108 17698 11107 159 1 0 0 smtp
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909509] [17708] 108 17708 11107 159 1 0 0 smtp
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909512] [17715] 108 17715 11107 159 0 0 0 smtp
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909515] [17719] 108 17719 11107 142 1 0 0 smtp
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909518] Out of memory: Kill process 16007 (php) score 858 or sacrifice child
Jul 15 18:32:51 ip-10-61-174-46 kernel: [1440066.909529] Killed process 16007 (php) total-vm:3541560kB, anon-rss:3288800kB, file-rss:0kB
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016126] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016130] php cpuset=/ mems_allowed=0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016133] Pid: 17929, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016134] Call Trace:
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016141] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016144] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016146] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016149] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016154] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016157] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016160] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016162] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016165] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016168] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016171] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016174] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016176] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016179] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016181] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016184] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016187] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016189] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016192] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016195] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016197] Mem-Info:
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016198] Node 0 DMA per-cpu:
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016200] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016201] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016202] Node 0 DMA32 per-cpu:
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016204] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016205] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016208] active_anon:932638 inactive_anon:1393 isolated_anon:0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016208] active_file:39 inactive_file:18 isolated_file:0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016209] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016210] free:4048 slab_reclaimable:2976 slab_unreclaimable:2586
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016210] mapped:1386 shmem:1405 pagetables:3065 bounce:0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016212] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016218] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016221] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730552kB inactive_anon:5572kB active_file:156kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5544kB shmem:5620kB slab_reclaimable:11904kB slab_unreclaimable:10344kB kernel_stack:728kB pagetables:12260kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6460 all_unreclaimable? yes
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016228] lowmem_reserve[]: 0 0 0 0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016230] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016237] Node 0 DMA32: 1437*4kB 19*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016244] 1480 total pagecache pages
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016245] 0 pages in swap cache
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016246] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016247] Free swap = 0kB
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.016248] Total swap = 0kB
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021551] 985072 pages RAM
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021553] 24998 pages reserved
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021554] 3817 pages shared
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021555] 953297 pages non-shared
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021557] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021567] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021570] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021574] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021577] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021580] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021584] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021587] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021590] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021594] [ 790] 101 790 63477 634 0 0 0 rsyslogd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021597] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021600] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021604] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021607] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021610] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021613] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021616] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021619] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021622] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021626] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021629] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021632] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021635] [ 658] 1012 658 134457 70863 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021638] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021641] [13468] 0 13468 89584 2502 0 0 0 php5-fpm
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021644] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021647] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021650] [ 2686] 1012 2686 68618 5669 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021654] [17929] 1012 17929 891683 828663 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021657] [18846] 108 18846 6792 70 1 0 0 pickup
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021660] [24804] 1012 24804 68580 5605 0 0 0 php
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021663] Out of memory: Kill process 17929 (php) score 864 or sacrifice child
Jul 15 18:47:43 ip-10-61-174-46 kernel: [1440959.021675] Killed process 17929 (php) total-vm:3566732kB, anon-rss:3314536kB, file-rss:116kB
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480103] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480107] php cpuset=/ mems_allowed=0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480110] Pid: 25020, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480112] Call Trace:
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480119] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480121] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480124] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480126] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480130] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480133] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480135] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480138] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480141] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480144] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480147] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480149] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480152] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480154] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480157] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480160] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480162] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480164] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480167] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480169] Mem-Info:
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480170] Node 0 DMA per-cpu:
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480172] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480173] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480175] Node 0 DMA32 per-cpu:
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480176] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480177] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480180] active_anon:932661 inactive_anon:1393 isolated_anon:0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480181] active_file:15 inactive_file:23 isolated_file:0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480182] unevictable:0 dirty:22 writeback:0 unstable:0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480182] free:4048 slab_reclaimable:2709 slab_unreclaimable:2637
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480183] mapped:1376 shmem:1405 pagetables:3247 bounce:0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480185] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480191] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480194] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730644kB inactive_anon:5572kB active_file:60kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5504kB shmem:5620kB slab_reclaimable:10836kB slab_unreclaimable:10548kB kernel_stack:792kB pagetables:12988kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9074 all_unreclaimable? yes
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480201] lowmem_reserve[]: 0 0 0 0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480203] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480210] Node 0 DMA32: 1240*4kB 106*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480216] 1487 total pagecache pages
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480218] 0 pages in swap cache
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480219] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480220] Free swap = 0kB
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.480221] Total swap = 0kB
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485396] 985072 pages RAM
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485398] 24998 pages reserved
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485399] 3806 pages shared
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485400] 953299 pages non-shared
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485403] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485412] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485415] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485418] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485421] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485424] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485428] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485431] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485435] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485438] [ 790] 101 790 63477 634 0 0 0 rsyslogd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485442] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485445] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485448] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485451] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485454] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485457] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485460] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485463] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485467] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485470] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485473] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485476] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485479] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485482] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485485] [13468] 0 13468 89584 2536 1 0 0 php5-fpm
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485488] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485491] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485494] [ 2686] 1012 2686 68618 5669 1 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485498] [25020] 1012 25020 869959 806834 0 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485501] [ 4134] 108 4134 6792 69 0 0 0 pickup
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485505] [ 8066] 108 8066 6818 74 1 0 0 cleanup
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485508] [ 8067] 108 8067 6795 70 0 0 0 trivial-rewrite
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485511] [ 8068] 108 8068 11107 157 0 0 0 smtp
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485514] [ 8075] 108 8075 11107 157 0 0 0 smtp
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485517] [ 8092] 108 8092 11107 158 1 0 0 smtp
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485520] [ 8098] 108 8098 11107 158 0 0 0 smtp
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485523] [ 8102] 108 8102 11107 159 1 0 0 smtp
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485525] [ 8106] 108 8106 6792 70 1 0 0 scache
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485529] [ 8305] 1012 8305 71448 8507 1 0 0 php
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485532] Out of memory: Kill process 25020 (php) score 842 or sacrifice child
Jul 15 19:21:27 ip-10-61-174-46 kernel: [1442983.485543] Killed process 25020 (php) total-vm:3479836kB, anon-rss:3227272kB, file-rss:64kB
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332185] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332189] php cpuset=/ mems_allowed=0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332192] Pid: 9857, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332193] Call Trace:
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332201] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332203] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332205] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332208] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332211] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332215] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332217] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332220] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332225] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332227] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332230] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332233] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332235] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332238] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332240] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332242] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332244] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332246] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332250] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332251] Mem-Info:
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332252] Node 0 DMA per-cpu:
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332254] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332256] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332257] Node 0 DMA32 per-cpu:
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332258] CPU 0: hi: 186, btch: 31 usd: 41
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332260] CPU 1: hi: 186, btch: 31 usd: 71
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332262] active_anon:932806 inactive_anon:1393 isolated_anon:0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332263] active_file:1 inactive_file:16 isolated_file:0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332264] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332264] free:4023 slab_reclaimable:2592 slab_unreclaimable:2608
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332265] mapped:1350 shmem:1405 pagetables:3212 bounce:0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332267] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332273] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332276] Node 0 DMA32 free:7756kB min:7848kB low:9808kB high:11772kB active_anon:3731224kB inactive_anon:5572kB active_file:4kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:10368kB slab_unreclaimable:10432kB kernel_stack:776kB pagetables:12848kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:161 all_unreclaimable? yes
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332283] lowmem_reserve[]: 0 0 0 0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332285] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332292] Node 0 DMA32: 809*4kB 155*8kB 29*16kB 18*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7692kB
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332299] 1438 total pagecache pages
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332300] 0 pages in swap cache
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332301] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332302] Free swap = 0kB
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.332303] Total swap = 0kB
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337569] 985072 pages RAM
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337571] 24998 pages reserved
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337572] 3773 pages shared
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337573] 953287 pages non-shared
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337575] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337584] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337588] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337591] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337594] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337597] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337600] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337604] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337607] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337611] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337614] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337617] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337621] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337624] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337627] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337630] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337633] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337636] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337639] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337643] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337646] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337649] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337652] [ 658] 1012 658 152314 88755 1 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337655] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337658] [13468] 0 13468 89584 2540 1 0 0 php5-fpm
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337661] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337664] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337667] [ 2686] 1012 2686 68618 5668 0 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337671] [ 4134] 108 4134 6792 75 1 0 0 pickup
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337674] [ 8447] 1012 8447 871831 808820 1 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337677] [ 8913] 108 8913 6818 73 0 0 0 cleanup
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337680] [ 8914] 108 8914 6795 70 1 0 0 trivial-rewrite
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337683] [ 8919] 108 8919 11107 159 0 0 0 smtp
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337686] [ 9794] 108 9794 11107 157 0 0 0 smtp
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337690] [ 9798] 108 9798 11107 158 0 0 0 smtp
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337693] [ 9802] 108 9802 11107 158 1 0 0 smtp
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337696] [ 9857] 1012 9857 69851 6879 1 0 0 php
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337698] Out of memory: Kill process 8447 (php) score 844 or sacrifice child
Jul 15 19:24:40 ip-10-61-174-46 kernel: [1443176.337709] Killed process 8447 (php) total-vm:3487324kB, anon-rss:3235280kB, file-rss:0kB
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016964] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016968] php cpuset=/ mems_allowed=0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016974] Pid: 9987, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016975] Call Trace:
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016982] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016985] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016987] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016990] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016993] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016997] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.016999] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017002] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017005] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017008] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017011] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017013] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017016] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017018] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017021] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017024] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017026] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017028] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017031] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017033] Mem-Info:
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017034] Node 0 DMA per-cpu:
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017036] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017037] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017038] Node 0 DMA32 per-cpu:
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017040] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017041] CPU 1: hi: 186, btch: 31 usd: 105
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017044] active_anon:933121 inactive_anon:1393 isolated_anon:0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017044] active_file:0 inactive_file:0 isolated_file:0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017045] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017045] free:4016 slab_reclaimable:2458 slab_unreclaimable:2564
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017046] mapped:1379 shmem:1405 pagetables:3069 bounce:0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017048] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017054] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017057] Node 0 DMA32 free:7728kB min:7848kB low:9808kB high:11772kB active_anon:3732484kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5620kB slab_reclaimable:9832kB slab_unreclaimable:10256kB kernel_stack:728kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:134 all_unreclaimable? yes
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017064] lowmem_reserve[]: 0 0 0 0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017066] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017073] Node 0 DMA32: 1154*4kB 133*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7728kB
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017080] 1451 total pagecache pages
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017081] 0 pages in swap cache
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017082] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017083] Free swap = 0kB
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.017084] Total swap = 0kB
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022402] 985072 pages RAM
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022404] 24998 pages reserved
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022405] 3784 pages shared
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022406] 953250 pages non-shared
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022409] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022418] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022421] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022424] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022428] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022431] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022434] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022437] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022441] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022445] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022448] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022451] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022454] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022457] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022460] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022463] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022466] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022470] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022473] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022476] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022479] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022482] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022485] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022488] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022491] [13468] 0 13468 89584 2525 0 0 0 php5-fpm
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022494] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022497] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022500] [ 2686] 1012 2686 68618 5668 1 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022504] [ 4134] 108 4134 6792 75 1 0 0 pickup
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022507] [ 9987] 1012 9987 874258 811210 0 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022510] [12444] 1012 12444 68580 5604 0 0 0 php
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022513] Out of memory: Kill process 9987 (php) score 846 or sacrifice child
Jul 15 19:30:11 ip-10-61-174-46 kernel: [1443507.022524] Killed process 9987 (php) total-vm:3497032kB, anon-rss:3244840kB, file-rss:0kB
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920458] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920462] php cpuset=/ mems_allowed=0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920465] Pid: 13422, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920466] Call Trace:
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920473] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920476] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920478] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920481] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920485] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920488] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920491] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920493] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920496] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920499] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920502] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920504] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920507] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920509] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920512] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920514] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920516] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920518] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920521] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920523] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920525] Mem-Info:
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920526] Node 0 DMA per-cpu:
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920528] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920529] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920530] Node 0 DMA32 per-cpu:
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920532] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920534] CPU 1: hi: 186, btch: 31 usd: 27
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920536] active_anon:932962 inactive_anon:1393 isolated_anon:0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920537] active_file:0 inactive_file:0 isolated_file:0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920537] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920538] free:4028 slab_reclaimable:2678 slab_unreclaimable:2553
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920539] mapped:1352 shmem:1405 pagetables:3079 bounce:0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920540] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920547] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920550] Node 0 DMA32 free:7776kB min:7848kB low:9808kB high:11772kB active_anon:3731848kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:10712kB slab_unreclaimable:10212kB kernel_stack:728kB pagetables:12316kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920556] lowmem_reserve[]: 0 0 0 0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920559] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920566] Node 0 DMA32: 1408*4kB 12*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7776kB
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920572] 1404 total pagecache pages
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920573] 0 pages in swap cache
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920575] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920576] Free swap = 0kB
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.920576] Total swap = 0kB
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925777] 985072 pages RAM
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925780] 24998 pages reserved
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925781] 3787 pages shared
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925782] 953313 pages non-shared
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925784] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925793] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925796] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925799] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925803] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925806] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925809] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925812] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925816] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925820] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925823] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925826] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925829] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925832] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925835] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925838] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925841] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925844] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925847] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925850] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925854] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925857] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925860] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925863] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925866] [13468] 0 13468 89584 2510 1 0 0 php5-fpm
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925869] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925872] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925875] [ 2686] 1012 2686 68618 5669 1 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925879] [ 4134] 108 4134 6792 75 1 0 0 pickup
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925882] [13422] 1012 13422 869768 806663 0 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925885] [17853] 1012 17853 72944 9989 1 0 0 php
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925888] Out of memory: Kill process 13422 (php) score 841 or sacrifice child
Jul 15 19:43:09 ip-10-61-174-46 kernel: [1444284.925899] Killed process 13422 (php) total-vm:3479072kB, anon-rss:3226652kB, file-rss:0kB
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956361] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956364] php cpuset=/ mems_allowed=0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956367] Pid: 18054, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956369] Call Trace:
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956376] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956378] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956380] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956383] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956386] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956390] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956392] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956395] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956397] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956401] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956403] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956406] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956408] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956411] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956414] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956416] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956418] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956421] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956424] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956426] Mem-Info:
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956427] Node 0 DMA per-cpu:
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956429] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956430] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956431] Node 0 DMA32 per-cpu:
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956433] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956434] CPU 1: hi: 186, btch: 31 usd: 31
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956437] active_anon:933052 inactive_anon:1393 isolated_anon:0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956438] active_file:0 inactive_file:0 isolated_file:0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956438] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956439] free:4036 slab_reclaimable:2501 slab_unreclaimable:2575
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956440] mapped:1358 shmem:1405 pagetables:3137 bounce:0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956441] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956448] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956451] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3732208kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:10004kB slab_unreclaimable:10300kB kernel_stack:752kB pagetables:12548kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? yes
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956457] lowmem_reserve[]: 0 0 0 0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956460] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956466] Node 0 DMA32: 646*4kB 151*8kB 41*16kB 39*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7808kB
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956473] 1438 total pagecache pages
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956474] 0 pages in swap cache
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956476] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956477] Free swap = 0kB
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.956478] Total swap = 0kB
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961714] 985072 pages RAM
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961716] 24998 pages reserved
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961717] 3778 pages shared
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961718] 953307 pages non-shared
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961720] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961729] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961732] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961735] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961739] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961742] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961745] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961748] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961752] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961755] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961759] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961762] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961765] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961768] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961771] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961774] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961777] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961780] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961784] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961787] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961790] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961793] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961796] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961799] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961803] [13468] 0 13468 89584 2495 1 0 0 php5-fpm
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961806] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961809] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961812] [ 2686] 1012 2686 68618 5668 1 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961815] [ 4134] 108 4134 6792 75 0 0 0 pickup
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961819] [18054] 1012 18054 867537 804450 0 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961822] [18290] 1012 18290 74971 11967 1 0 0 php
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961825] [18363] 108 18363 6818 72 1 0 0 cleanup
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961828] [18364] 108 18364 6795 71 1 0 0 trivial-rewrite
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961831] [18365] 108 18365 11107 159 1 0 0 smtp
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961834] Out of memory: Kill process 18054 (php) score 839 or sacrifice child
Jul 15 19:44:09 ip-10-61-174-46 kernel: [1444344.961845] Killed process 18054 (php) total-vm:3470148kB, anon-rss:3217800kB, file-rss:0kB
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996126] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996131] php cpuset=/ mems_allowed=0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996134] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996136] Call Trace:
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996144] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996147] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996154] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996158] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996163] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996167] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996170] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996174] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996177] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996181] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996184] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996187] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996190] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996195] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996202] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996206] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996209] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996213] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996216] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996218] Mem-Info:
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996220] Node 0 DMA per-cpu:
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996222] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996226] Node 0 DMA32 per-cpu:
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996228] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996230] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996233] active_anon:932753 inactive_anon:1393 isolated_anon:0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996234] active_file:6 inactive_file:8 isolated_file:0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996235] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996236] free:4048 slab_reclaimable:2684 slab_unreclaimable:2562
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996237] mapped:1369 shmem:1405 pagetables:3187 bounce:0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996239] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996248] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996252] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731012kB inactive_anon:5572kB active_file:24kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:10736kB slab_unreclaimable:10248kB kernel_stack:768kB pagetables:12748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1179 all_unreclaimable? yes
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996260] lowmem_reserve[]: 0 0 0 0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996264] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996273] Node 0 DMA32: 1342*4kB 63*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7968kB
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996282] 1444 total pagecache pages
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996283] 0 pages in swap cache
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996285] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996286] Free swap = 0kB
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444848.996287] Total swap = 0kB
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001470] 985072 pages RAM
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001472] 24998 pages reserved
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001473] 3783 pages shared
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001474] 953321 pages non-shared
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001476] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001485] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001488] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001491] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001494] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001497] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001501] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001504] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001508] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001511] [ 790] 101 790 63477 636 0 0 0 rsyslogd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001515] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001518] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001521] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001524] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001527] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001530] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001533] [ 1051] 0 1051 201627 844 0 0 0 collectd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001536] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001539] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001543] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001546] [ 654] 1012 654 70351 5762 1 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001549] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001552] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001555] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001558] [13468] 0 13468 89584 2519 0 0 0 php5-fpm
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001561] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001564] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001567] [ 2686] 1012 2686 68618 5668 1 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001571] [ 4134] 108 4134 6792 75 1 0 0 pickup
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001574] [18457] 1012 18457 873335 810197 1 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001577] [21804] 108 21804 6818 72 1 0 0 cleanup
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001580] [21805] 108 21805 6795 71 0 0 0 trivial-rewrite
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001583] [21806] 108 21806 11107 159 1 0 0 smtp
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001586] [21810] 108 21810 11107 159 0 0 0 smtp
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001589] [21814] 108 21814 11107 158 1 0 0 smtp
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001592] [21889] 1012 21889 68580 5605 0 0 0 php
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001595] Out of memory: Kill process 18457 (php) score 845 or sacrifice child
Jul 15 19:52:33 ip-10-61-174-46 kernel: [1444849.001606] Killed process 18457 (php) total-vm:3493340kB, anon-rss:3240788kB, file-rss:0kB
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056117] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056122] php cpuset=/ mems_allowed=0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056124] Pid: 22093, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056126] Call Trace:
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056133] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056135] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056137] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056140] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056146] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056148] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056154] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056156] [<ffffffff8111a6c1>] ? filemap_fault+0x1/0x3c0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056159] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056162] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056165] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056168] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056170] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056173] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056175] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056178] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056181] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056183] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056185] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056188] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056191] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056192] Mem-Info:
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056193] Node 0 DMA per-cpu:
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056195] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056197] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056198] Node 0 DMA32 per-cpu:
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056199] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056201] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056203] active_anon:933414 inactive_anon:1393 isolated_anon:0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056204] active_file:3 inactive_file:10 isolated_file:0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056205] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056205] free:4048 slab_reclaimable:2234 slab_unreclaimable:2582
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056206] mapped:1366 shmem:1405 pagetables:3065 bounce:0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056207] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056214] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056217] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733656kB inactive_anon:5572kB active_file:12kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:8936kB slab_unreclaimable:10328kB kernel_stack:728kB pagetables:12260kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3110 all_unreclaimable? yes
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056224] lowmem_reserve[]: 0 0 0 0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056226] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056233] Node 0 DMA32: 803*4kB 122*8kB 82*16kB 13*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7964kB
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056240] 1411 total pagecache pages
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056241] 0 pages in swap cache
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056242] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056243] Free swap = 0kB
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.056244] Total swap = 0kB
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061420] 985072 pages RAM
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061422] 24998 pages reserved
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061423] 3792 pages shared
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061424] 953318 pages non-shared
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061426] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061436] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061439] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061442] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061445] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061449] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061452] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061455] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061458] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061462] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061466] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061469] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061472] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061475] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061478] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061481] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061484] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061487] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061490] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061494] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061497] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061500] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061503] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061506] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061510] [13468] 0 13468 89584 2504 1 0 0 php5-fpm
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061513] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061516] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061519] [ 2686] 1012 2686 68618 5668 1 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061523] [ 4134] 108 4134 6792 75 0 0 0 pickup
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061526] [22093] 1012 22093 874578 811506 0 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061529] [22527] 1012 22527 68580 5605 1 0 0 php
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061532] Out of memory: Kill process 22093 (php) score 846 or sacrifice child
Jul 15 19:53:38 ip-10-61-174-46 kernel: [1444914.061544] Killed process 22093 (php) total-vm:3498312kB, anon-rss:3246024kB, file-rss:0kB
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872114] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872118] php cpuset=/ mems_allowed=0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872121] Pid: 22656, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872122] Call Trace:
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872129] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872131] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872134] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872137] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872141] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872144] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872148] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872150] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872153] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872155] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872158] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872161] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872163] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872165] [<ffffffff811425e4>] ? find_vma+0x34/0x70
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872169] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872171] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872174] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872177] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872178] Mem-Info:
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872180] Node 0 DMA per-cpu:
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872181] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872183] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872184] Node 0 DMA32 per-cpu:
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872185] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872187] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872189] active_anon:933204 inactive_anon:1393 isolated_anon:0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872190] active_file:0 inactive_file:0 isolated_file:0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872191] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872191] free:4048 slab_reclaimable:2349 slab_unreclaimable:2587
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872192] mapped:1335 shmem:1405 pagetables:3072 bounce:0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872194] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872200] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872203] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732816kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5340kB shmem:5620kB slab_reclaimable:9396kB slab_unreclaimable:10348kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872209] lowmem_reserve[]: 0 0 0 0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872212] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872219] Node 0 DMA32: 952*4kB 238*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872225] 1456 total pagecache pages
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872226] 0 pages in swap cache
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872228] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872229] Free swap = 0kB
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.872230] Total swap = 0kB
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877449] 985072 pages RAM
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877451] 24998 pages reserved
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877453] 3798 pages shared
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877454] 953336 pages non-shared
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877455] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877464] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877468] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877471] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877474] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877477] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877481] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877484] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877488] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877492] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877495] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877498] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877501] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877504] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877507] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877510] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877513] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877516] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877519] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877523] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877526] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877529] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877532] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877535] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877538] [13468] 0 13468 89584 2525 0 0 0 php5-fpm
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877541] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877544] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877547] [ 2686] 1012 2686 68618 5668 0 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877551] [ 4134] 108 4134 6792 75 0 0 0 pickup
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877554] [22656] 1012 22656 874386 811299 0 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877557] [23258] 1012 23258 68580 5604 1 0 0 php
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877560] Out of memory: Kill process 22656 (php) score 846 or sacrifice child
Jul 15 19:55:43 ip-10-61-174-46 kernel: [1445038.877571] Killed process 22656 (php) total-vm:3497544kB, anon-rss:3245196kB, file-rss:0kB
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817027] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817031] php cpuset=/ mems_allowed=0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817033] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817035] Call Trace:
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817042] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817045] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817047] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817050] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817054] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817057] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817060] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817062] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817065] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817068] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817071] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817073] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817076] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817079] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817081] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817083] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817087] [<ffffffff8106bc3c>] ? do_wait+0x12c/0x260
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817090] [<ffffffff8106cf4b>] ? sys_wait4+0xab/0xf0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817092] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817094] Mem-Info:
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817095] Node 0 DMA per-cpu:
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817097] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817098] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817099] Node 0 DMA32 per-cpu:
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817101] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817102] CPU 1: hi: 186, btch: 31 usd: 29
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817105] active_anon:932967 inactive_anon:1393 isolated_anon:0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817106] active_file:13 inactive_file:0 isolated_file:0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817106] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817107] free:4029 slab_reclaimable:2477 slab_unreclaimable:2589
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817108] mapped:1365 shmem:1405 pagetables:3187 bounce:0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817109] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817116] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817119] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3731868kB inactive_anon:5572kB active_file:52kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:9908kB slab_unreclaimable:10356kB kernel_stack:768kB pagetables:12748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:220 all_unreclaimable? yes
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817125] lowmem_reserve[]: 0 0 0 0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817127] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817134] Node 0 DMA32: 640*4kB 106*8kB 72*16kB 35*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817141] 1433 total pagecache pages
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817142] 0 pages in swap cache
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817143] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817145] Free swap = 0kB
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.817145] Total swap = 0kB
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822279] 985072 pages RAM
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822282] 24998 pages reserved
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822283] 3782 pages shared
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822284] 953291 pages non-shared
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822287] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822295] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822299] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822302] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822305] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822308] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822311] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822315] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822318] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822322] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822325] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822328] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822332] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822335] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822338] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822341] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822344] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822347] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822350] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822354] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822356] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822359] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822362] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822366] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822369] [13468] 0 13468 89584 2526 0 0 0 php5-fpm
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822372] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822375] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822378] [ 2686] 1012 2686 68618 5668 0 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822382] [ 4134] 108 4134 6792 75 1 0 0 pickup
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822385] [23353] 108 23353 6818 74 1 0 0 cleanup
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822388] [23354] 108 23354 6795 70 1 0 0 trivial-rewrite
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822391] [23355] 108 23355 11107 159 0 0 0 smtp
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822394] [23361] 108 23361 11107 159 0 0 0 smtp
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822397] [23366] 108 23366 11107 158 1 0 0 smtp
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822400] [23429] 1012 23429 873490 810410 0 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822403] [23505] 1012 23505 68580 5604 0 0 0 php
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822406] Out of memory: Kill process 23429 (php) score 845 or sacrifice child
Jul 15 19:56:22 ip-10-61-174-46 kernel: [1445077.822417] Killed process 23429 (php) total-vm:3493960kB, anon-rss:3241640kB, file-rss:0kB
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284146] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284150] php cpuset=/ mems_allowed=0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284153] Pid: 24247, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284154] Call Trace:
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284161] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284164] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284166] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284169] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284174] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284177] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284185] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284188] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284191] [<ffffffff8100633d>] ? pte_mfn_to_pfn+0x6d/0x110
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284196] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284201] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284203] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284206] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284211] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284213] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284216] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284218] Mem-Info:
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284219] Node 0 DMA per-cpu:
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284221] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284222] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284223] Node 0 DMA32 per-cpu:
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284225] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284226] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284229] active_anon:932707 inactive_anon:1393 isolated_anon:0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284229] active_file:2 inactive_file:0 isolated_file:0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284230] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284231] free:4048 slab_reclaimable:2774 slab_unreclaimable:2633
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284231] mapped:1366 shmem:1405 pagetables:3159 bounce:0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284233] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284240] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284242] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730828kB inactive_anon:5572kB active_file:8kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:11096kB slab_unreclaimable:10532kB kernel_stack:752kB pagetables:12636kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:19 all_unreclaimable? yes
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284249] lowmem_reserve[]: 0 0 0 0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284251] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284258] Node 0 DMA32: 1378*4kB 32*8kB 6*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7944kB
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284265] 1453 total pagecache pages
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284266] 0 pages in swap cache
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284267] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284268] Free swap = 0kB
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.284269] Total swap = 0kB
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289513] 985072 pages RAM
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289515] 24998 pages reserved
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289516] 3784 pages shared
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289517] 953328 pages non-shared
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289519] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289527] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289531] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289534] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289537] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289540] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289543] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289547] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289550] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289554] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289557] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289560] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289564] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289567] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289570] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289573] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289576] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289579] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289582] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289585] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289589] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289592] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289595] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289598] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289601] [13468] 0 13468 89584 2535 0 0 0 php5-fpm
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289604] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289607] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289610] [ 2686] 1012 2686 68618 5669 1 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289614] [24247] 1012 24247 873443 810330 0 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289616] [24260] 108 24260 6792 69 1 0 0 pickup
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289620] [25376] 108 25376 6818 73 1 0 0 cleanup
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289623] [25377] 108 25377 6795 69 1 0 0 trivial-rewrite
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289626] [26926] 108 26926 11107 158 1 0 0 smtp
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289629] [26930] 108 26930 11107 157 1 0 0 smtp
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289632] [28170] 1012 28170 68580 5605 0 0 0 php
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289634] Out of memory: Kill process 24247 (php) score 845 or sacrifice child
Jul 15 20:05:11 ip-10-61-174-46 kernel: [1445607.289645] Killed process 24247 (php) total-vm:3493772kB, anon-rss:3241320kB, file-rss:0kB
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156099] php invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156103] php cpuset=/ mems_allowed=0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156106] Pid: 1297, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156107] Call Trace:
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156114] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156116] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156119] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156122] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156125] [<ffffffff8111804a>] ? unlock_page+0x2a/0x40
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156128] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156131] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156134] [<ffffffff81661f78>] ? do_page_fault+0x288/0x550
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156136] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156138] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156141] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156144] [<ffffffff81177941>] ? fd_install+0x61/0x80
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156147] [<ffffffff81183c33>] ? do_pipe_flags+0xc3/0x120
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156150] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156153] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156156] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156157] Mem-Info:
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156158] Node 0 DMA per-cpu:
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156160] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156162] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156163] Node 0 DMA32 per-cpu:
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156164] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156166] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156168] active_anon:932651 inactive_anon:1393 isolated_anon:0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156169] active_file:0 inactive_file:0 isolated_file:0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156170] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156170] free:4241 slab_reclaimable:2534 slab_unreclaimable:2664
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156171] mapped:1358 shmem:1405 pagetables:3254 bounce:0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156172] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156179] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156182] Node 0 DMA32 free:8628kB min:7848kB low:9808kB high:11772kB active_anon:3730604kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:10136kB slab_unreclaimable:10656kB kernel_stack:800kB pagetables:13016kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:17 all_unreclaimable? yes
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156188] lowmem_reserve[]: 0 0 0 0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156191] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156198] Node 0 DMA32: 1417*4kB 88*8kB 13*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8628kB
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156204] 1428 total pagecache pages
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156205] 0 pages in swap cache
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156207] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156208] Free swap = 0kB
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.156209] Total swap = 0kB
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161413] 985072 pages RAM
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161416] 24998 pages reserved
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161417] 3784 pages shared
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161418] 953154 pages non-shared
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161420] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161428] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161432] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161435] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161438] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161441] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161445] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161448] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161451] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161455] [ 790] 101 790 63477 636 0 0 0 rsyslogd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161458] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161461] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161464] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161468] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161471] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161474] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161477] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161480] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161483] [ 1242] 107 1242 9445 161 1 0 0 ntpd
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161486] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161489] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161492] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161495] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161499] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161502] [13468] 0 13468 89584 2503 1 0 0 php5-fpm
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161505] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161508] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161511] [ 2686] 1012 2686 68618 5669 1 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161514] [24260] 108 24260 6792 75 0 0 0 pickup
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161518] [29987] 1012 29987 867319 804340 0 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161521] [32355] 108 32355 6818 74 1 0 0 cleanup
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161524] [32356] 108 32356 6795 71 0 0 0 trivial-rewrite
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161527] [32357] 108 32357 11107 160 1 0 0 smtp
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161530] [32361] 108 32361 11107 159 1 0 0 smtp
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161533] [ 1111] 1012 1111 68580 5605 0 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161536] [ 1296] 1012 1296 1099 25 0 0 0 sh
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161539] [ 1297] 1012 1297 68825 5889 0 0 0 php
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161542] Out of memory: Kill process 29987 (php) score 839 or sacrifice child
Jul 15 20:15:02 ip-10-61-174-46 kernel: [1446198.161553] Killed process 29987 (php) total-vm:3469276kB, anon-rss:3217356kB, file-rss:4kB
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748113] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748117] php cpuset=/ mems_allowed=0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748120] Pid: 1362, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748122] Call Trace:
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748128] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748131] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748133] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748136] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748141] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748144] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748147] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748149] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748152] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748156] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748159] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748162] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748165] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748172] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748174] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748177] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748179] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748181] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748183] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748186] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748188] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748190] Mem-Info:
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748191] Node 0 DMA per-cpu:
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748193] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748194] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748195] Node 0 DMA32 per-cpu:
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748197] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748198] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748201] active_anon:933035 inactive_anon:1393 isolated_anon:0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748201] active_file:11 inactive_file:11 isolated_file:0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748202] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748203] free:4048 slab_reclaimable:2398 slab_unreclaimable:2639
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748203] mapped:1366 shmem:1405 pagetables:3174 bounce:0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748205] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748211] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748214] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732140kB inactive_anon:5572kB active_file:44kB inactive_file:44kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9592kB slab_unreclaimable:10556kB kernel_stack:768kB pagetables:12696kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2548 all_unreclaimable? yes
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748221] lowmem_reserve[]: 0 0 0 0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748223] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748230] Node 0 DMA32: 1148*4kB 162*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748236] 1449 total pagecache pages
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748238] 0 pages in swap cache
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748239] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748240] Free swap = 0kB
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.748241] Total swap = 0kB
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753369] 985072 pages RAM
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753371] 24998 pages reserved
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753373] 3793 pages shared
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753374] 953315 pages non-shared
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753376] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753384] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753388] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753391] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753394] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753397] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753401] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753404] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753408] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753412] [ 790] 101 790 63477 636 0 0 0 rsyslogd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753415] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753418] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753421] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753424] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753427] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753430] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753433] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753437] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753440] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753443] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753446] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753449] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753452] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753455] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753458] [13468] 0 13468 89584 2507 0 0 0 php5-fpm
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753461] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753464] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753468] [ 2686] 1012 2686 68618 5668 1 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753471] [24260] 108 24260 6792 75 0 0 0 pickup
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753475] [ 1362] 1012 1362 870549 807509 0 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753478] [ 3627] 108 3627 6818 74 0 0 0 cleanup
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753481] [ 3628] 108 3628 6795 70 1 0 0 trivial-rewrite
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753484] [ 3629] 108 3629 11107 159 0 0 0 smtp
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753487] [ 3633] 108 3633 11107 157 0 0 0 smtp
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753490] [ 3637] 108 3637 11107 159 0 0 0 smtp
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753493] [ 4115] 1012 4115 71499 8548 0 0 0 php
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753496] Out of memory: Kill process 1362 (php) score 842 or sacrifice child
Jul 15 20:19:10 ip-10-61-174-46 kernel: [1446445.753507] Killed process 1362 (php) total-vm:3482196kB, anon-rss:3230036kB, file-rss:0kB
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536874] cron invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536878] cron cpuset=/ mems_allowed=0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536880] Pid: 15657, comm: cron Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536882] Call Trace:
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536889] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536892] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536894] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536897] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536900] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536903] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536905] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536909] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536912] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536914] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536917] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536919] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536921] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536926] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536929] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536933] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536935] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536938] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536940] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536943] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536945] Mem-Info:
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536946] Node 0 DMA per-cpu:
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536948] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536949] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536950] Node 0 DMA32 per-cpu:
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536952] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536953] CPU 1: hi: 186, btch: 31 usd: 28
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536956] active_anon:933271 inactive_anon:1393 isolated_anon:0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536957] active_file:18 inactive_file:0 isolated_file:10
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536957] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536958] free:4043 slab_reclaimable:2306 slab_unreclaimable:2586
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536959] mapped:1362 shmem:1405 pagetables:3074 bounce:0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536960] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536967] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536970] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3733084kB inactive_anon:5572kB active_file:72kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):40kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:9224kB slab_unreclaimable:10344kB kernel_stack:728kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:238 all_unreclaimable? yes
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536976] lowmem_reserve[]: 0 0 0 0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536979] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536985] Node 0 DMA32: 1213*4kB 109*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536992] 1438 total pagecache pages
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536993] 0 pages in swap cache
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536995] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536996] Free swap = 0kB
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.536997] Total swap = 0kB
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542202] 985072 pages RAM
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542205] 24998 pages reserved
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542206] 3779 pages shared
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542207] 953305 pages non-shared
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542209] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542218] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542221] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542224] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542227] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542230] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542234] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542237] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542241] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542244] [ 790] 101 790 63477 636 0 0 0 rsyslogd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542248] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542251] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542254] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542257] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542260] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542264] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542267] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542270] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542273] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542276] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542279] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542282] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542285] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542288] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542291] [13468] 0 13468 89584 2521 1 0 0 php5-fpm
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542294] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542297] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542300] [ 2686] 1012 2686 68618 5668 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542304] [24260] 108 24260 6792 75 0 0 0 pickup
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542308] [ 4397] 1012 4397 874354 811345 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542311] [ 6091] 1012 6091 68580 5604 0 0 0 php
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542314] Out of memory: Kill process 4397 (php) score 846 or sacrifice child
Jul 15 20:22:02 ip-10-61-174-46 kernel: [1446617.542325] Killed process 4397 (php) total-vm:3497416kB, anon-rss:3245380kB, file-rss:0kB
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216196] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216200] php cpuset=/ mems_allowed=0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216203] Pid: 6348, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216204] Call Trace:
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216212] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216214] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216216] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216219] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216224] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216226] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216230] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216232] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216235] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216237] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216240] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216242] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216245] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216248] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216250] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216253] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216255] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216257] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216260] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216263] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216265] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216267] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216270] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216272] Mem-Info:
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216273] Node 0 DMA per-cpu:
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216275] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216276] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216277] Node 0 DMA32 per-cpu:
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216279] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216280] CPU 1: hi: 186, btch: 31 usd: 62
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216283] active_anon:932643 inactive_anon:1393 isolated_anon:0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216284] active_file:22 inactive_file:19 isolated_file:0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216285] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216285] free:4038 slab_reclaimable:2647 slab_unreclaimable:2658
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216286] mapped:1355 shmem:1405 pagetables:3230 bounce:0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216287] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216294] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216297] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3730572kB inactive_anon:5572kB active_file:88kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10588kB slab_unreclaimable:10632kB kernel_stack:784kB pagetables:12920kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:286 all_unreclaimable? yes
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216303] lowmem_reserve[]: 0 0 0 0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216306] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216313] Node 0 DMA32: 1448*4kB 2*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216319] 1408 total pagecache pages
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216321] 0 pages in swap cache
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216322] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216323] Free swap = 0kB
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.216324] Total swap = 0kB
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221502] 985072 pages RAM
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221504] 24998 pages reserved
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221505] 3775 pages shared
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221506] 953263 pages non-shared
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221508] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221517] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221520] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221524] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221527] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221530] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221534] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221537] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221540] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221544] [ 790] 101 790 63477 636 1 0 0 rsyslogd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221547] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221551] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221554] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221557] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221560] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221563] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221566] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221570] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221573] [ 1242] 107 1242 9445 161 0 0 0 ntpd
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221576] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221579] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221582] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221585] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221588] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221591] [13468] 0 13468 89584 2534 0 0 0 php5-fpm
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221594] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221598] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221601] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221604] [ 6348] 1012 6348 872986 809761 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221607] [ 6521] 108 6521 6795 71 1 0 0 trivial-rewrite
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221611] [ 7723] 108 7723 6818 74 0 0 0 cleanup
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221614] [ 7732] 108 7732 11107 160 0 0 0 smtp
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221617] [10193] 108 10193 6792 69 0 0 0 pickup
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221620] [10197] 108 10197 11107 159 0 0 0 smtp
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221623] [10201] 108 10201 11107 158 1 0 0 smtp
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221626] [11012] 108 11012 11107 158 1 0 0 smtp
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221629] [11016] 108 11016 11107 159 1 0 0 smtp
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221632] [11314] 1012 11314 68580 5604 1 0 0 php
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221635] Out of memory: Kill process 6348 (php) score 845 or sacrifice child
Jul 15 20:29:54 ip-10-61-174-46 kernel: [1447090.221646] Killed process 6348 (php) total-vm:3491944kB, anon-rss:3239044kB, file-rss:0kB
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492179] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492183] php cpuset=/ mems_allowed=0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492186] Pid: 11512, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492187] Call Trace:
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492195] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492197] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492199] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492202] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492205] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492209] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492211] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492214] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492218] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492221] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492223] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492225] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492228] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492230] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492233] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492236] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492238] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492240] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492242] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492244] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492247] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492249] Mem-Info:
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492250] Node 0 DMA per-cpu:
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492252] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492253] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492254] Node 0 DMA32 per-cpu:
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492256] CPU 0: hi: 186, btch: 31 usd: 32
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492257] CPU 1: hi: 186, btch: 31 usd: 22
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492259] active_anon:932672 inactive_anon:1393 isolated_anon:0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492260] active_file:34 inactive_file:30 isolated_file:0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492261] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492261] free:4025 slab_reclaimable:2573 slab_unreclaimable:2653
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492262] mapped:1355 shmem:1405 pagetables:3286 bounce:0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492264] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492270] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492273] Node 0 DMA32 free:7764kB min:7848kB low:9808kB high:11772kB active_anon:3730688kB inactive_anon:5572kB active_file:136kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10292kB slab_unreclaimable:10612kB kernel_stack:784kB pagetables:13144kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:483 all_unreclaimable? yes
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492280] lowmem_reserve[]: 0 0 0 0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492282] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492289] Node 0 DMA32: 1397*4kB 16*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7764kB
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492295] 1442 total pagecache pages
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492297] 0 pages in swap cache
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492298] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492299] Free swap = 0kB
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.492300] Total swap = 0kB
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497500] 985072 pages RAM
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497502] 24998 pages reserved
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497504] 3786 pages shared
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497505] 953313 pages non-shared
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497507] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497516] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497519] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497522] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497526] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497529] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497532] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497535] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497539] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497543] [ 790] 101 790 63477 636 0 0 0 rsyslogd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497546] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497549] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497553] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497556] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497559] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497562] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497565] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497568] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497572] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497575] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497578] [ 654] 1012 654 70351 5762 0 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497581] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497584] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497587] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497590] [13468] 0 13468 89584 2538 0 0 0 php5-fpm
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497593] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497596] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497599] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497603] [10193] 108 10193 6792 75 0 0 0 pickup
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497606] [11512] 1012 11512 869717 806447 1 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497609] [14415] 108 14415 6818 73 0 0 0 cleanup
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497612] [14416] 108 14416 6795 71 0 0 0 trivial-rewrite
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497615] [14417] 108 14417 11107 158 0 0 0 smtp
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497618] [14421] 108 14421 11107 159 0 0 0 smtp
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497622] [14425] 108 14425 11107 158 1 0 0 smtp
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497625] [15224] 1012 15224 68580 5603 0 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497628] [15437] 1012 15437 1099 25 0 0 0 sh
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497631] [15438] 1012 15438 66161 3636 1 0 0 php
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497633] Out of memory: Kill process 11512 (php) score 841 or sacrifice child
Jul 15 20:35:30 ip-10-61-174-46 kernel: [1447426.497645] Killed process 11512 (php) total-vm:3478868kB, anon-rss:3225784kB, file-rss:4kB
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208300] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208304] php cpuset=/ mems_allowed=0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208306] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208308] Call Trace:
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208315] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208318] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208320] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208323] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208326] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208329] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208331] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208335] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208338] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208340] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208343] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208346] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208348] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208351] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208354] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208358] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208360] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208363] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208366] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208368] Mem-Info:
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208369] Node 0 DMA per-cpu:
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208371] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208372] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208373] Node 0 DMA32 per-cpu:
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208375] CPU 0: hi: 186, btch: 31 usd: 40
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208376] CPU 1: hi: 186, btch: 31 usd: 60
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208379] active_anon:932893 inactive_anon:1393 isolated_anon:0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208380] active_file:27 inactive_file:0 isolated_file:0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208380] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208381] free:4034 slab_reclaimable:2429 slab_unreclaimable:2654
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208382] mapped:1366 shmem:1405 pagetables:3194 bounce:0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208383] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208390] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208393] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3731572kB inactive_anon:5572kB active_file:108kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9716kB slab_unreclaimable:10616kB kernel_stack:776kB pagetables:12776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:382 all_unreclaimable? yes
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208399] lowmem_reserve[]: 0 0 0 0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208402] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208409] Node 0 DMA32: 1232*4kB 103*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208415] 1424 total pagecache pages
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208417] 0 pages in swap cache
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208418] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208419] Free swap = 0kB
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.208420] Total swap = 0kB
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213573] 985072 pages RAM
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213575] 24998 pages reserved
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213576] 3775 pages shared
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213578] 953270 pages non-shared
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213579] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213588] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213591] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213595] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213598] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213601] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213604] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213607] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213611] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213615] [ 790] 101 790 63477 636 0 0 0 rsyslogd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213618] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213621] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213625] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213628] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213631] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213634] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213637] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213640] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213643] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213646] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213650] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213653] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213656] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213659] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213662] [13468] 0 13468 89584 2523 1 0 0 php5-fpm
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213665] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213668] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213671] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213675] [10193] 108 10193 6792 75 1 0 0 pickup
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213678] [15470] 1012 15470 868740 805501 0 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213681] [21867] 108 21867 6792 69 0 0 0 scache
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213684] [23137] 108 23137 6818 73 0 0 0 cleanup
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213687] [23138] 108 23138 6795 70 1 0 0 trivial-rewrite
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213690] [23139] 108 23139 11107 160 0 0 0 smtp
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213693] [23143] 108 23143 11107 157 0 0 0 smtp
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213696] [23147] 108 23147 11107 159 0 0 0 smtp
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213699] [24413] 1012 24413 73309 10374 1 0 0 php
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213702] Out of memory: Kill process 15470 (php) score 840 or sacrifice child
Jul 15 20:48:55 ip-10-61-174-46 kernel: [1448231.213713] Killed process 15470 (php) total-vm:3474960kB, anon-rss:3222004kB, file-rss:0kB
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809672] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809677] php cpuset=/ mems_allowed=0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809681] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809683] Call Trace:
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809691] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809694] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809697] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809701] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809707] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809711] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809715] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809718] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809721] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809725] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809728] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809731] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809733] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809736] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809739] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809741] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809743] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809746] [<ffffffff811798c0>] ? vfs_write+0x110/0x180
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809749] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809751] Mem-Info:
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809752] Node 0 DMA per-cpu:
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809754] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809755] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809756] Node 0 DMA32 per-cpu:
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809758] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809759] CPU 1: hi: 186, btch: 31 usd: 25
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809762] active_anon:932604 inactive_anon:1393 isolated_anon:0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809762] active_file:7 inactive_file:0 isolated_file:0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809763] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809764] free:4048 slab_reclaimable:2683 slab_unreclaimable:2666
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809764] mapped:1363 shmem:1405 pagetables:3269 bounce:0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809766] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809772] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809775] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730416kB inactive_anon:5572kB active_file:28kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:10732kB slab_unreclaimable:10664kB kernel_stack:776kB pagetables:13076kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:73 all_unreclaimable? yes
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809782] lowmem_reserve[]: 0 0 0 0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809784] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809791] Node 0 DMA32: 1406*4kB 17*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809798] 1450 total pagecache pages
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809799] 0 pages in swap cache
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809800] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809801] Free swap = 0kB
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.809802] Total swap = 0kB
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814961] 985072 pages RAM
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814964] 24998 pages reserved
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814965] 3775 pages shared
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814966] 953301 pages non-shared
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814968] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814977] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814980] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814984] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814987] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814990] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814993] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.814996] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815000] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815004] [ 790] 101 790 63477 638 0 0 0 rsyslogd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815007] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815010] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815013] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815016] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815019] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815023] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815026] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815029] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815032] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815035] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815039] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815042] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815045] [ 658] 1012 658 152314 88755 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815048] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815051] [13468] 0 13468 89584 2527 0 0 0 php5-fpm
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815054] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815057] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815060] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815064] [10193] 108 10193 6792 75 1 0 0 pickup
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815067] [24566] 1012 24566 867708 804528 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815070] [28300] 108 28300 6818 74 1 0 0 cleanup
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815074] [28301] 108 28301 6795 70 0 0 0 trivial-rewrite
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815077] [28302] 108 28302 11107 158 1 0 0 smtp
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815080] [28696] 108 28696 6800 70 0 0 0 bounce
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815083] [29821] 108 29821 11076 141 1 0 0 smtp
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815086] [29920] 1012 29920 68580 5603 1 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815089] [30015] 1012 30015 68580 5604 0 0 0 php
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815092] Out of memory: Kill process 24566 (php) score 839 or sacrifice child
Jul 15 20:57:28 ip-10-61-174-46 kernel: [1448743.815102] Killed process 24566 (php) total-vm:3470832kB, anon-rss:3218112kB, file-rss:0kB
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808317] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808322] php cpuset=/ mems_allowed=0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808325] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808327] Call Trace:
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808335] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808338] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808341] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808345] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808349] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808354] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808357] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808361] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808366] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808370] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808374] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808378] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808381] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808385] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808388] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808392] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808395] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808400] [<ffffffff8106bc3c>] ? do_wait+0x12c/0x260
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808403] [<ffffffff8106cf4b>] ? sys_wait4+0xab/0xf0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808407] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808409] Mem-Info:
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808411] Node 0 DMA per-cpu:
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808413] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808415] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808417] Node 0 DMA32 per-cpu:
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808419] CPU 0: hi: 186, btch: 31 usd: 38
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808421] CPU 1: hi: 186, btch: 31 usd: 30
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808424] active_anon:932598 inactive_anon:1393 isolated_anon:0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808425] active_file:0 inactive_file:0 isolated_file:0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808426] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808427] free:4034 slab_reclaimable:2753 slab_unreclaimable:2623
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808428] mapped:1340 shmem:1405 pagetables:3206 bounce:0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808430] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808438] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808441] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3730392kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5360kB shmem:5620kB slab_reclaimable:11012kB slab_unreclaimable:10492kB kernel_stack:776kB pagetables:12824kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:77 all_unreclaimable? yes
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808448] lowmem_reserve[]: 0 0 0 0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808450] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808457] Node 0 DMA32: 1232*4kB 101*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808464] 1478 total pagecache pages
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808465] 0 pages in swap cache
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808466] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808467] Free swap = 0kB
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.808468] Total swap = 0kB
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813650] 985072 pages RAM
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813652] 24998 pages reserved
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813653] 3797 pages shared
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813654] 953280 pages non-shared
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813656] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813665] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813668] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813671] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813675] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813678] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813681] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813684] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813688] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813692] [ 790] 101 790 63477 638 0 0 0 rsyslogd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813695] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813698] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813702] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813705] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813708] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813711] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813714] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813717] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813721] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813724] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813727] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813730] [ 657] 1012 657 69242 6260 0 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813733] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813736] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813739] [13468] 0 13468 89584 2523 0 0 0 php5-fpm
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813742] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813745] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813748] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813752] [28300] 108 28300 6818 82 0 0 0 cleanup
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813756] [28301] 108 28301 6795 74 0 0 0 trivial-rewrite
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813759] [28302] 108 28302 11107 159 1 0 0 smtp
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813762] [29821] 108 29821 11107 158 1 0 0 smtp
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813765] [30116] 1012 30116 866936 803828 1 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813768] [31359] 108 31359 11107 159 1 0 0 smtp
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813771] [31441] 108 31441 6800 70 1 0 0 bounce
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813774] [32473] 108 32473 6792 70 0 0 0 pickup
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813777] [32523] 1012 32523 74663 11753 0 0 0 php
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813780] Out of memory: Kill process 30116 (php) score 838 or sacrifice child
Jul 15 21:01:27 ip-10-61-174-46 kernel: [1448982.813792] Killed process 30116 (php) total-vm:3467744kB, anon-rss:3215312kB, file-rss:0kB
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812139] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812144] php5-fpm cpuset=/ mems_allowed=0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812147] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812150] Call Trace:
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812161] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812164] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812167] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812171] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812176] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812178] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812185] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812189] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812191] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812194] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812196] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812199] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812204] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812208] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812212] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812214] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812216] Mem-Info:
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812217] Node 0 DMA per-cpu:
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812219] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812220] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812222] Node 0 DMA32 per-cpu:
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812223] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812224] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812227] active_anon:932541 inactive_anon:1393 isolated_anon:0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812228] active_file:0 inactive_file:0 isolated_file:0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812228] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812229] free:4048 slab_reclaimable:2714 slab_unreclaimable:2625
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812230] mapped:1343 shmem:1405 pagetables:3334 bounce:0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812231] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812238] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812241] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730164kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5372kB shmem:5620kB slab_reclaimable:10856kB slab_unreclaimable:10500kB kernel_stack:800kB pagetables:13336kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812247] lowmem_reserve[]: 0 0 0 0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812250] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812256] Node 0 DMA32: 1246*4kB 107*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812263] 1445 total pagecache pages
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812264] 0 pages in swap cache
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812266] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812267] Free swap = 0kB
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.812268] Total swap = 0kB
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817584] 985072 pages RAM
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817587] 24998 pages reserved
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817588] 3778 pages shared
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817590] 953328 pages non-shared
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817591] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817599] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817603] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817606] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817609] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817612] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817616] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817619] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817622] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817626] [ 790] 101 790 63477 638 0 0 0 rsyslogd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817629] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817632] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817636] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817639] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817642] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817645] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817648] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817651] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817654] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817658] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817661] [ 654] 1012 654 70351 5765 0 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817664] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817667] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817670] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817673] [13468] 0 13468 89584 2508 1 0 0 php5-fpm
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817676] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817679] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817682] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817686] [28300] 108 28300 6818 82 1 0 0 cleanup
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817689] [28301] 108 28301 6795 74 1 0 0 trivial-rewrite
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817692] [28302] 108 28302 11107 158 0 0 0 smtp
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817695] [29821] 108 29821 11107 157 0 0 0 smtp
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817698] [31359] 108 31359 11107 160 0 0 0 smtp
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817701] [32473] 108 32473 6792 76 1 0 0 pickup
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817704] [32643] 1012 32643 81725 18820 0 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817707] [ 947] 108 947 11107 158 0 0 0 smtp
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817710] [ 951] 108 951 11107 158 1 0 0 smtp
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817713] [ 954] 1012 954 853536 790515 0 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817716] [ 1161] 1012 1161 1099 24 1 0 0 sh
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817719] [ 1162] 1012 1162 68825 5870 0 0 0 php
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817722] Out of memory: Kill process 954 (php) score 825 or sacrifice child
Jul 15 21:03:00 ip-10-61-174-46 kernel: [1449075.817734] Killed process 954 (php) total-vm:3414144kB, anon-rss:3162060kB, file-rss:0kB
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804951] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804956] php cpuset=/ mems_allowed=0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804959] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804961] Call Trace:
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804970] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804973] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804976] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804980] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804984] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804990] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804995] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.804998] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805001] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805005] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805009] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805013] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805016] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805019] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805024] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805028] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805031] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805035] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805038] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805042] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805045] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805048] Mem-Info:
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805049] Node 0 DMA per-cpu:
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805052] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805054] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805055] Node 0 DMA32 per-cpu:
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805057] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805059] CPU 1: hi: 186, btch: 31 usd: 11
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805063] active_anon:933106 inactive_anon:1393 isolated_anon:0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805064] active_file:0 inactive_file:13 isolated_file:0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805065] unevictable:0 dirty:17 writeback:0 unstable:0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805066] free:4021 slab_reclaimable:2502 slab_unreclaimable:2576
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805067] mapped:1348 shmem:1405 pagetables:3075 bounce:0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805069] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805077] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805081] Node 0 DMA32 free:7748kB min:7848kB low:9808kB high:11772kB active_anon:3732424kB inactive_anon:5572kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:68kB writeback:0kB mapped:5392kB shmem:5620kB slab_reclaimable:10008kB slab_unreclaimable:10304kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:195 all_unreclaimable? yes
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805089] lowmem_reserve[]: 0 0 0 0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805092] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805102] Node 0 DMA32: 839*4kB 227*8kB 27*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7748kB
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805111] 1441 total pagecache pages
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805113] 0 pages in swap cache
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805115] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805116] Free swap = 0kB
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.805117] Total swap = 0kB
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810332] 985072 pages RAM
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810334] 24998 pages reserved
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810336] 3775 pages shared
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810337] 953349 pages non-shared
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810339] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810347] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810351] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810354] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810357] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810360] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810363] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810367] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810370] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810374] [ 790] 101 790 63477 638 1 0 0 rsyslogd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810377] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810380] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810384] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810387] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810390] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810393] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810396] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810399] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810402] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810406] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810409] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810411] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810414] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810417] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810420] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810423] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810426] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810429] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810433] [32473] 108 32473 6792 76 1 0 0 pickup
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810436] [32643] 1012 32643 874151 811167 0 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810439] [13455] 1012 13455 68580 5604 0 0 0 php
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810442] Out of memory: Kill process 32643 (php) score 846 or sacrifice child
Jul 15 21:21:30 ip-10-61-174-46 kernel: [1450185.810453] Killed process 32643 (php) total-vm:3496604kB, anon-rss:3244668kB, file-rss:0kB
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896138] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896142] php cpuset=/ mems_allowed=0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896145] Pid: 15580, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896146] Call Trace:
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896158] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896161] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896164] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896168] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896172] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896177] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896180] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896182] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896185] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896187] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896190] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896194] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896196] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896199] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896202] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896204] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896207] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896210] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896211] Mem-Info:
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896213] Node 0 DMA per-cpu:
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896214] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896216] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896217] Node 0 DMA32 per-cpu:
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896219] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896220] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896222] active_anon:933153 inactive_anon:1393 isolated_anon:0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896223] active_file:0 inactive_file:0 isolated_file:0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896224] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896224] free:4048 slab_reclaimable:2457 slab_unreclaimable:2573
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896225] mapped:1357 shmem:1405 pagetables:3071 bounce:0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896233] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732612kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9828kB slab_unreclaimable:10292kB kernel_stack:728kB pagetables:12284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896242] lowmem_reserve[]: 0 0 0 0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896252] Node 0 DMA32: 773*4kB 188*8kB 65*16kB 7*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7908kB
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896258] 1465 total pagecache pages
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896260] 0 pages in swap cache
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896261] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896262] Free swap = 0kB
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.896263] Total swap = 0kB
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901911] 985072 pages RAM
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901914] 24998 pages reserved
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901916] 3808 pages shared
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901917] 953316 pages non-shared
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901919] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901927] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901931] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901934] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901937] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901940] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901944] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901947] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901950] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901954] [ 790] 101 790 63477 638 1 0 0 rsyslogd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901957] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901961] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901964] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901967] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901970] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901973] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901976] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901979] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901982] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901986] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901989] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901992] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901995] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.901998] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902001] [13468] 0 13468 89584 2504 0 0 0 php5-fpm
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902004] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902007] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902011] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902014] [32473] 108 32473 6792 76 0 0 0 pickup
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902017] [13531] 1012 13531 874224 811215 0 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902020] [15580] 1012 15580 68580 5604 1 0 0 php
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902023] Out of memory: Kill process 13531 (php) score 846 or sacrifice child
Jul 15 21:24:39 ip-10-61-174-46 kernel: [1450374.902033] Killed process 13531 (php) total-vm:3496896kB, anon-rss:3244860kB, file-rss:0kB
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204139] php cpuset=/ mems_allowed=0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204142] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204144] Call Trace:
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204150] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204153] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204155] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204158] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204163] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204169] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204171] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204174] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204178] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204181] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204184] [<ffffffff81006311>] ? pte_mfn_to_pfn+0x41/0x110
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204187] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204189] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204191] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204194] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204196] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204200] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204203] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204206] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204208] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204210] Mem-Info:
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204211] Node 0 DMA per-cpu:
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204214] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204216] Node 0 DMA32 per-cpu:
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204217] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204221] active_anon:933077 inactive_anon:1393 isolated_anon:0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204222] active_file:0 inactive_file:0 isolated_file:0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204222] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204223] free:4048 slab_reclaimable:2550 slab_unreclaimable:2604
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204224] mapped:1345 shmem:1405 pagetables:3021 bounce:0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204225] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204232] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204235] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732308kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5380kB shmem:5620kB slab_reclaimable:10200kB slab_unreclaimable:10416kB kernel_stack:752kB pagetables:12084kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204241] lowmem_reserve[]: 0 0 0 0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204244] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204251] Node 0 DMA32: 757*4kB 125*8kB 51*16kB 31*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204257] 1473 total pagecache pages
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204258] 0 pages in swap cache
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204260] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204261] Free swap = 0kB
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.204262] Total swap = 0kB
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209520] 985072 pages RAM
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209522] 24998 pages reserved
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209523] 3815 pages shared
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209525] 953299 pages non-shared
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209527] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209535] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209539] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209542] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209545] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209548] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209552] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209555] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209558] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209562] [ 790] 101 790 63477 638 1 0 0 rsyslogd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209565] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209568] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209571] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209575] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209578] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209581] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209584] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209587] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209591] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209594] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209597] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209600] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209603] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209606] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209609] [13468] 0 13468 89584 2512 1 0 0 php5-fpm
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209612] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209615] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209618] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209622] [32473] 108 32473 6792 76 0 0 0 pickup
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209625] [15766] 1012 15766 879633 816428 0 0 0 php
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209628] [16176] 108 16176 6818 73 0 0 0 cleanup
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209631] [16178] 108 16178 6795 69 1 0 0 trivial-rewrite
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209634] [16180] 108 16180 11107 159 1 0 0 smtp
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209638] Out of memory: Kill process 15766 (php) score 852 or sacrifice child
Jul 15 21:25:49 ip-10-61-174-46 kernel: [1450445.209649] Killed process 15766 (php) total-vm:3518532kB, anon-rss:3265672kB, file-rss:40kB
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116004] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116008] ntpd cpuset=/ mems_allowed=0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116011] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116013] Call Trace:
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116020] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116023] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116025] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116028] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116031] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116034] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116037] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116039] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116042] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116045] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116048] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116052] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116069] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116073] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116075] Mem-Info:
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116077] Node 0 DMA per-cpu:
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116080] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116081] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116083] Node 0 DMA32 per-cpu:
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116085] CPU 0: hi: 186, btch: 31 usd: 13
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116087] CPU 1: hi: 186, btch: 31 usd: 83
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116091] active_anon:932957 inactive_anon:1393 isolated_anon:0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116092] active_file:10 inactive_file:16 isolated_file:19
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116093] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116094] free:4019 slab_reclaimable:2599 slab_unreclaimable:2575
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116095] mapped:1356 shmem:1405 pagetables:3076 bounce:0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116097] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116105] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116109] Node 0 DMA32 free:7740kB min:7848kB low:9808kB high:11772kB active_anon:3731828kB inactive_anon:5572kB active_file:40kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10396kB slab_unreclaimable:10300kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:256 all_unreclaimable? yes
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116118] lowmem_reserve[]: 0 0 0 0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116122] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116132] Node 0 DMA32: 1423*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7740kB
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116142] 1449 total pagecache pages
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116143] 0 pages in swap cache
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116145] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116147] Free swap = 0kB
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.116148] Total swap = 0kB
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121490] 985072 pages RAM
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121492] 24998 pages reserved
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121493] 3785 pages shared
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121494] 953275 pages non-shared
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121496] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121503] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121506] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121508] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121510] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121513] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121515] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121517] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121520] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121523] [ 790] 101 790 63477 638 0 0 0 rsyslogd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121525] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121527] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121530] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121532] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121534] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121536] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121539] [ 1051] 0 1051 201629 844 1 0 0 collectd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121541] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121543] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121545] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121547] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121550] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121552] [ 658] 1012 658 152314 88755 1 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121554] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121556] [13468] 0 13468 89584 2492 1 0 0 php5-fpm
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121559] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121561] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121563] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121566] [32473] 108 32473 6792 76 1 0 0 pickup
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121568] [16408] 1012 16408 874173 811017 1 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121570] [19272] 1012 19272 68580 5603 1 0 0 php
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121573] Out of memory: Kill process 16408 (php) score 846 or sacrifice child
Jul 15 21:30:14 ip-10-61-174-46 kernel: [1450709.121582] Killed process 16408 (php) total-vm:3496692kB, anon-rss:3244068kB, file-rss:0kB
Jul 15 21:31:24 ip-10-61-174-46 kernel: [1450780.508151] trivial-rewrite invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508156] trivial-rewrite cpuset=/ mems_allowed=0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508159] Pid: 19649, comm: trivial-rewrite Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508161] Call Trace:
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508169] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508172] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508174] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508177] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508180] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508184] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508186] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508189] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508192] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508195] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508198] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508203] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508206] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508209] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508213] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508217] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508220] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508222] Mem-Info:
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508223] Node 0 DMA per-cpu:
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508225] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508226] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508227] Node 0 DMA32 per-cpu:
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508229] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508230] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508233] active_anon:932527 inactive_anon:1393 isolated_anon:0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508233] active_file:0 inactive_file:7 isolated_file:0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508234] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508235] free:4047 slab_reclaimable:2660 slab_unreclaimable:2656
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508235] mapped:1358 shmem:1405 pagetables:3288 bounce:0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508237] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508244] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508247] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3730108kB inactive_anon:5572kB active_file:0kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:10640kB slab_unreclaimable:10624kB kernel_stack:808kB pagetables:13152kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6298 all_unreclaimable? yes
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508253] lowmem_reserve[]: 0 0 0 0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508256] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508262] Node 0 DMA32: 868*4kB 145*8kB 36*16kB 20*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8024kB
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508269] 1419 total pagecache pages
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508270] 0 pages in swap cache
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508272] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508273] Free swap = 0kB
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.508273] Total swap = 0kB
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513454] 985072 pages RAM
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513456] 24998 pages reserved
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513457] 3796 pages shared
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513458] 953296 pages non-shared
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513460] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513468] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513472] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513475] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513478] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513481] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513485] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513488] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513492] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513495] [ 790] 101 790 63477 638 1 0 0 rsyslogd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513499] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513502] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513505] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513508] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513511] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513514] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513517] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513521] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513524] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513527] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513530] [ 654] 1012 654 70351 5763 0 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513533] [ 657] 1012 657 69242 6264 1 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513536] [ 658] 1012 658 152314 88755 0 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513539] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513542] [13468] 0 13468 89584 2516 1 0 0 php5-fpm
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513545] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513548] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513551] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513555] [32473] 108 32473 6792 76 0 0 0 pickup
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513559] [19518] 1012 19518 872396 809293 0 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513562] [19648] 108 19648 6818 73 0 0 0 cleanup
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513565] [19649] 108 19649 6795 70 0 0 0 trivial-rewrite
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513568] [19650] 108 19650 11107 158 0 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513571] [19654] 108 19654 11107 159 0 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513574] [19658] 108 19658 11107 156 0 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513577] [19684] 108 19684 11107 158 1 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513579] [19688] 108 19688 11107 159 0 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513582] [19695] 108 19695 11108 159 1 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513585] [19696] 108 19696 6792 69 0 0 0 scache
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513588] [19712] 108 19712 11108 160 1 0 0 smtp
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513592] [20065] 1012 20065 68580 5605 1 0 0 php
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513594] Out of memory: Kill process 19518 (php) score 844 or sacrifice child
Jul 15 21:31:25 ip-10-61-174-46 kernel: [1450780.513606] Killed process 19518 (php) total-vm:3489584kB, anon-rss:3237172kB, file-rss:0kB
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488158] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488162] php cpuset=/ mems_allowed=0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488165] Pid: 27896, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488167] Call Trace:
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488174] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488176] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488178] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488181] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488184] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488188] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488190] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488193] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488197] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488200] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488202] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488205] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488207] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488210] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488213] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488215] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488217] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488219] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488221] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488224] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488226] Mem-Info:
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488227] Node 0 DMA per-cpu:
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488228] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488230] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488231] Node 0 DMA32 per-cpu:
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488232] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488234] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488236] active_anon:932581 inactive_anon:1393 isolated_anon:0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488237] active_file:0 inactive_file:0 isolated_file:0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488238] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488238] free:4048 slab_reclaimable:2643 slab_unreclaimable:2651
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488239] mapped:1366 shmem:1405 pagetables:3313 bounce:0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488240] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488247] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488250] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730324kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:10572kB slab_unreclaimable:10604kB kernel_stack:800kB pagetables:13252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488256] lowmem_reserve[]: 0 0 0 0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488259] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488266] Node 0 DMA32: 1224*4kB 104*8kB 5*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488272] 1450 total pagecache pages
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488273] 0 pages in swap cache
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488275] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488276] Free swap = 0kB
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.488277] Total swap = 0kB
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494055] 985072 pages RAM
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494058] 24998 pages reserved
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494059] 3808 pages shared
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494060] 953322 pages non-shared
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494061] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494070] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494073] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494076] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494079] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494082] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494085] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494088] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494091] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494095] [ 790] 101 790 63477 638 1 0 0 rsyslogd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494098] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494101] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494104] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494107] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494110] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494113] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494116] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494119] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494122] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494125] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494129] [ 654] 1012 654 70351 5763 1 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494132] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494135] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494138] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494141] [13468] 0 13468 89584 2540 0 0 0 php5-fpm
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494144] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494147] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494150] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494153] [32473] 108 32473 6792 76 0 0 0 pickup
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494157] [20884] 1012 20884 867122 803947 0 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494160] [26022] 108 26022 6818 74 0 0 0 cleanup
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494163] [26023] 108 26023 6795 71 1 0 0 trivial-rewrite
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494166] [26028] 108 26028 11107 158 0 0 0 smtp
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494169] [26032] 108 26032 11107 158 0 0 0 smtp
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494172] [26040] 108 26040 11107 158 0 0 0 smtp
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494175] [27896] 1012 27896 68580 5604 1 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494178] [27991] 1012 27991 1099 25 1 0 0 sh
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494181] [27992] 1012 27992 69046 6021 0 0 0 php
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494184] [28019] 1012 28019 1099 27 0 0 0 sh
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494187] Out of memory: Kill process 20884 (php) score 839 or sacrifice child
Jul 15 21:43:11 ip-10-61-174-46 kernel: [1451486.494198] Killed process 20884 (php) total-vm:3468488kB, anon-rss:3215788kB, file-rss:0kB
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540124] ssh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540128] ssh cpuset=/ mems_allowed=0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540131] Pid: 3079, comm: ssh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540133] Call Trace:
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540139] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540152] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540154] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540158] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540160] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540163] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540167] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540170] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540172] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540175] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540177] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540180] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540183] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540185] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540187] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540191] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540193] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540195] Mem-Info:
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540196] Node 0 DMA per-cpu:
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540198] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540199] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540200] Node 0 DMA32 per-cpu:
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540202] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540203] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540206] active_anon:932747 inactive_anon:1393 isolated_anon:0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540206] active_file:0 inactive_file:0 isolated_file:0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540207] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540208] free:4048 slab_reclaimable:2460 slab_unreclaimable:2631
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540209] mapped:1353 shmem:1405 pagetables:3314 bounce:0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540210] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540217] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540220] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730988kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:9840kB slab_unreclaimable:10524kB kernel_stack:768kB pagetables:13256kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540226] lowmem_reserve[]: 0 0 0 0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540228] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540235] Node 0 DMA32: 1352*4kB 44*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540242] 1460 total pagecache pages
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540243] 0 pages in swap cache
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540244] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540245] Free swap = 0kB
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.540246] Total swap = 0kB
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545421] 985072 pages RAM
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545423] 24998 pages reserved
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545424] 3812 pages shared
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545425] 953317 pages non-shared
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545427] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545436] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545440] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545443] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545446] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545449] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545452] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545456] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545459] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545463] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545466] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545469] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545472] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545475] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545478] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545482] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545485] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545488] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545491] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545494] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545497] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545500] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545503] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545506] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545510] [13468] 0 13468 89584 2538 0 0 0 php5-fpm
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545513] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545516] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545519] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545522] [28083] 1012 28083 862278 799109 0 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545526] [31385] 108 31385 6792 70 1 0 0 pickup
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545529] [ 2803] 1012 2803 68580 5604 0 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545532] [ 3067] 1012 3067 1099 24 1 0 0 sh
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545535] [ 3068] 1012 3068 68825 5872 1 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545538] [ 3074] 1012 3074 1099 25 1 0 0 sh
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545541] [ 3075] 1012 3075 4654 59 1 0 0 git
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545544] [ 3076] 1012 3076 68499 5551 1 0 0 php
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545547] [ 3078] 1012 3078 1099 25 1 0 0 sh
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545550] [ 3079] 1012 3079 10901 175 0 0 0 ssh
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545553] Out of memory: Kill process 28083 (php) score 834 or sacrifice child
Jul 15 21:53:41 ip-10-61-174-46 kernel: [1452116.545564] Killed process 28083 (php) total-vm:3449112kB, anon-rss:3196436kB, file-rss:0kB
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804145] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804149] php cpuset=/ mems_allowed=0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804152] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804153] Call Trace:
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804161] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804163] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804165] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804168] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804173] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804176] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804185] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804188] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804190] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804192] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804195] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804197] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804201] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804205] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804208] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804211] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804214] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804217] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804219] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804222] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804224] Mem-Info:
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804225] Node 0 DMA per-cpu:
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804227] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804228] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804229] Node 0 DMA32 per-cpu:
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804231] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804232] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804235] active_anon:932915 inactive_anon:1393 isolated_anon:0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804236] active_file:0 inactive_file:9 isolated_file:0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804236] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804237] free:4048 slab_reclaimable:2512 slab_unreclaimable:2621
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804237] mapped:1377 shmem:1405 pagetables:3185 bounce:0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804239] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804246] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804249] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731660kB inactive_anon:5572kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:10048kB slab_unreclaimable:10484kB kernel_stack:776kB pagetables:12740kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:82 all_unreclaimable? yes
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804255] lowmem_reserve[]: 0 0 0 0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804258] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804264] Node 0 DMA32: 813*4kB 242*8kB 26*16kB 7*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804271] 1468 total pagecache pages
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804272] 0 pages in swap cache
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804273] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804275] Free swap = 0kB
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.804275] Total swap = 0kB
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809532] 985072 pages RAM
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809535] 24998 pages reserved
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809536] 3782 pages shared
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809537] 953328 pages non-shared
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809539] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809548] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809551] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809554] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809557] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809561] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809564] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809567] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809571] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809575] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809578] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809581] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809584] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809587] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809590] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809593] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809596] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809599] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809602] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809606] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809609] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809612] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809615] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809618] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809621] [13468] 0 13468 89584 2508 1 0 0 php5-fpm
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809624] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809627] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809631] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809634] [31385] 108 31385 6792 76 0 0 0 pickup
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809638] [ 3220] 1012 3220 873380 810194 0 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809641] [ 3285] 108 3285 6818 73 0 0 0 cleanup
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809644] [ 3286] 108 3286 6795 71 1 0 0 trivial-rewrite
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809647] [ 3287] 108 3287 11107 159 0 0 0 smtp
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809650] [ 3295] 108 3295 11107 159 0 0 0 smtp
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809653] [ 3333] 108 3333 11107 159 1 0 0 smtp
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809656] [ 4062] 108 4062 11107 158 0 0 0 smtp
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809659] [ 4521] 1012 4521 68580 5605 1 0 0 php
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809662] Out of memory: Kill process 3220 (php) score 845 or sacrifice child
Jul 15 21:55:47 ip-10-61-174-46 kernel: [1452242.809674] Killed process 3220 (php) total-vm:3493520kB, anon-rss:3240776kB, file-rss:0kB
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640126] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640130] php cpuset=/ mems_allowed=0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640132] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640134] Call Trace:
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640141] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640143] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640146] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640148] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640153] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640156] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640159] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640162] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640165] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640167] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640169] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640172] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640175] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640178] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640180] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640182] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640186] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640189] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640191] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640194] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640195] Mem-Info:
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640197] Node 0 DMA per-cpu:
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640198] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640200] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640201] Node 0 DMA32 per-cpu:
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640202] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640204] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640206] active_anon:932476 inactive_anon:1393 isolated_anon:0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640207] active_file:15 inactive_file:17 isolated_file:0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640208] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640208] free:4048 slab_reclaimable:2779 slab_unreclaimable:2631
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640209] mapped:1400 shmem:1405 pagetables:3295 bounce:0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640210] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640217] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640220] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729904kB inactive_anon:5572kB active_file:60kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5600kB shmem:5620kB slab_reclaimable:11116kB slab_unreclaimable:10524kB kernel_stack:784kB pagetables:13180kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:38270 all_unreclaimable? yes
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640226] lowmem_reserve[]: 0 0 0 0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640229] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640235] Node 0 DMA32: 1000*4kB 214*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640242] 1457 total pagecache pages
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640243] 0 pages in swap cache
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640245] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640246] Free swap = 0kB
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.640247] Total swap = 0kB
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645674] 985072 pages RAM
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645677] 24998 pages reserved
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645678] 3819 pages shared
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645679] 953311 pages non-shared
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645681] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645690] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645693] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645697] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645700] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645703] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645707] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645710] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645713] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645717] [ 790] 101 790 63477 639 1 0 0 rsyslogd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645720] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645723] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645727] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645730] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645733] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645736] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645739] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645742] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645746] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645749] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645752] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645755] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645758] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645761] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645764] [13468] 0 13468 89584 2515 0 0 0 php5-fpm
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645768] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645771] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645774] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645778] [31385] 108 31385 6792 76 0 0 0 pickup
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645781] [ 3285] 108 3285 6818 81 1 0 0 cleanup
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645784] [ 3286] 108 3286 6795 75 1 0 0 trivial-rewrite
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645787] [ 3287] 108 3287 11107 158 1 0 0 smtp
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645790] [ 3295] 108 3295 11107 159 1 0 0 smtp
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645793] [ 3333] 108 3333 11107 159 1 0 0 smtp
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645797] [ 4062] 108 4062 11107 158 1 0 0 smtp
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645800] [ 4766] 1012 4766 857668 794517 0 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645802] [ 4815] 1012 4815 78100 15212 0 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645805] [ 5183] 1012 5183 68580 5604 0 0 0 php
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645808] Out of memory: Kill process 4766 (php) score 829 or sacrifice child
Jul 15 21:56:23 ip-10-61-174-46 kernel: [1452278.645820] Killed process 4766 (php) total-vm:3430672kB, anon-rss:3177980kB, file-rss:88kB
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960145] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960149] php cpuset=/ mems_allowed=0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960152] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960153] Call Trace:
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960160] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960162] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960164] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960167] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960174] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960176] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960179] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960182] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960184] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960187] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960189] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960196] [<ffffffff81645116>] ? mm_fault_error+0xec/0xf3
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960198] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960201] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960205] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960209] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960212] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960214] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960217] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960219] Mem-Info:
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960220] Node 0 DMA per-cpu:
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960222] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960223] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960224] Node 0 DMA32 per-cpu:
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960226] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960227] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960230] active_anon:932302 inactive_anon:1393 isolated_anon:0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960230] active_file:0 inactive_file:0 isolated_file:0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960231] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960232] free:4048 slab_reclaimable:2985 slab_unreclaimable:2644
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960232] mapped:1356 shmem:1405 pagetables:3253 bounce:0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960240] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960243] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729208kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:11940kB slab_unreclaimable:10576kB kernel_stack:792kB pagetables:13012kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960250] lowmem_reserve[]: 0 0 0 0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960252] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960259] Node 0 DMA32: 1444*4kB 14*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960266] 1443 total pagecache pages
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960267] 0 pages in swap cache
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960268] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960269] Free swap = 0kB
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.960270] Total swap = 0kB
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965463] 985072 pages RAM
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965466] 24998 pages reserved
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965467] 3822 pages shared
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965468] 953291 pages non-shared
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965470] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965479] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965483] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965486] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965489] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965492] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965496] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965499] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965502] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965506] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965510] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965513] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965516] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965519] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965522] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965525] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965528] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965532] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965535] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965538] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965541] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965544] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965547] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965550] [ 678] 1012 678 70152 5581 1 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965553] [13468] 0 13468 89584 2500 0 0 0 php5-fpm
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965557] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965560] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965563] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965566] [31385] 108 31385 6792 76 0 0 0 pickup
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965570] [ 6090] 1012 6090 865359 802234 1 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965573] [ 9676] 108 9676 6818 73 1 0 0 cleanup
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965576] [ 9677] 108 9677 6795 71 0 0 0 trivial-rewrite
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965579] [ 9678] 108 9678 11107 159 1 0 0 smtp
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965582] [10622] 108 10622 11107 159 0 0 0 smtp
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965585] [10629] 108 10629 11107 159 0 0 0 smtp
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965588] [10633] 108 10633 11108 159 0 0 0 smtp
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965591] [10634] 108 10634 6792 69 0 0 0 scache
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965594] [11681] 108 11681 11107 157 0 0 0 smtp
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965597] [12715] 1012 12715 75623 12731 0 0 0 php
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965600] Out of memory: Kill process 6090 (php) score 837 or sacrifice child
Jul 15 22:06:48 ip-10-61-174-46 kernel: [1452903.965611] Killed process 6090 (php) total-vm:3461436kB, anon-rss:3208936kB, file-rss:0kB
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856168] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856172] php cpuset=/ mems_allowed=0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856174] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856176] Call Trace:
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856183] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856185] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856188] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856191] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856194] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856197] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856200] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856202] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856205] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856208] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856211] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856213] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856217] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856220] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856223] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856226] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856229] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856231] Mem-Info:
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856232] Node 0 DMA per-cpu:
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856234] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856235] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856236] Node 0 DMA32 per-cpu:
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856238] CPU 0: hi: 186, btch: 31 usd: 53
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856239] CPU 1: hi: 186, btch: 31 usd: 2
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856242] active_anon:932618 inactive_anon:1393 isolated_anon:0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856243] active_file:18 inactive_file:6 isolated_file:12
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856243] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856244] free:4035 slab_reclaimable:2672 slab_unreclaimable:2672
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856245] mapped:1353 shmem:1405 pagetables:3227 bounce:0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856246] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856253] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856256] Node 0 DMA32 free:7804kB min:7848kB low:9808kB high:11772kB active_anon:3730472kB inactive_anon:5572kB active_file:72kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10688kB slab_unreclaimable:10688kB kernel_stack:784kB pagetables:12908kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:221 all_unreclaimable? yes
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856262] lowmem_reserve[]: 0 0 0 0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856265] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856272] Node 0 DMA32: 1334*4kB 56*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7832kB
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856278] 1432 total pagecache pages
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856279] 0 pages in swap cache
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856281] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856282] Free swap = 0kB
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.856283] Total swap = 0kB
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861564] 985072 pages RAM
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861566] 24998 pages reserved
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861568] 3774 pages shared
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861569] 953307 pages non-shared
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861571] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861580] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861583] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861586] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861590] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861593] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861596] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861599] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861603] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861607] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861610] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861613] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861616] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861619] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861622] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861626] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861629] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861632] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861635] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861638] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861641] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861645] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861648] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861651] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861654] [13468] 0 13468 89584 2522 1 0 0 php5-fpm
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861657] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861660] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861663] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861667] [13098] 1012 13098 870311 807207 0 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861670] [17508] 108 17508 6792 70 0 0 0 pickup
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861673] [18137] 108 18137 6818 74 0 0 0 cleanup
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861676] [18138] 108 18138 6795 71 1 0 0 trivial-rewrite
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861679] [18245] 108 18245 11107 160 0 0 0 smtp
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861682] [18257] 108 18257 6792 70 0 0 0 scache
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861685] [19948] 108 19948 11107 142 0 0 0 smtp
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861688] [19952] 108 19952 11107 159 0 0 0 smtp
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861691] [19958] 108 19958 11076 136 0 0 0 smtp
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861694] [20003] 1012 20003 71151 8228 1 0 0 php
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861697] Out of memory: Kill process 13098 (php) score 842 or sacrifice child
Jul 15 22:16:30 ip-10-61-174-46 kernel: [1453485.861708] Killed process 13098 (php) total-vm:3481244kB, anon-rss:3228828kB, file-rss:0kB
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664127] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664131] php cpuset=/ mems_allowed=0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664134] Pid: 20326, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664136] Call Trace:
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664142] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664145] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664147] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664150] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664153] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664156] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664159] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664162] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664164] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664167] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664169] [<ffffffff811425f9>] ? find_vma+0x49/0x70
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664173] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664175] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664177] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664180] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664183] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664184] Mem-Info:
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664185] Node 0 DMA per-cpu:
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664187] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664189] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664190] Node 0 DMA32 per-cpu:
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664191] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664193] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664195] active_anon:932709 inactive_anon:1393 isolated_anon:0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664196] active_file:34 inactive_file:37 isolated_file:0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664196] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664197] free:4048 slab_reclaimable:2738 slab_unreclaimable:2635
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664198] mapped:1365 shmem:1405 pagetables:3184 bounce:0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664199] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664206] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664209] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730836kB inactive_anon:5572kB active_file:136kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:10952kB slab_unreclaimable:10540kB kernel_stack:760kB pagetables:12736kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15523 all_unreclaimable? yes
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664215] lowmem_reserve[]: 0 0 0 0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664218] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664225] Node 0 DMA32: 1302*4kB 59*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664231] 1471 total pagecache pages
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664233] 0 pages in swap cache
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664234] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664235] Free swap = 0kB
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.664236] Total swap = 0kB
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669366] 985072 pages RAM
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669368] 24998 pages reserved
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669369] 3801 pages shared
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669371] 953332 pages non-shared
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669373] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669381] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669385] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669388] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669391] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669394] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669398] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669401] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669404] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669408] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669411] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669414] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669418] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669421] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669424] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669427] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669430] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669433] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669437] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669440] [15657] 0 15657 4777 60 0 0 0 cron
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669443] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669446] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669449] [ 658] 1012 658 152314 88756 0 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669452] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669455] [13468] 0 13468 89584 2507 1 0 0 php5-fpm
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669458] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669461] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669464] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669468] [17508] 108 17508 6792 76 0 0 0 pickup
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669471] [18257] 108 18257 6792 70 1 0 0 scache
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669474] [20326] 1012 20326 870347 807252 0 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669477] [21780] 108 21780 6818 73 0 0 0 cleanup
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669480] [21781] 108 21781 6795 70 0 0 0 trivial-rewrite
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669484] [21782] 108 21782 11107 159 1 0 0 smtp
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669487] [21786] 108 21786 11107 157 1 0 0 smtp
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669489] [21979] 1012 21979 71484 8579 1 0 0 php
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669492] Out of memory: Kill process 20326 (php) score 842 or sacrifice child
Jul 15 22:19:09 ip-10-61-174-46 kernel: [1453644.669503] Killed process 20326 (php) total-vm:3481388kB, anon-rss:3228936kB, file-rss:72kB
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717280] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717284] collectd cpuset=/ mems_allowed=0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717287] Pid: 1075, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717289] Call Trace:
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717296] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717298] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717301] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717304] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717307] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717310] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717312] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717315] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717320] [<ffffffff8165e56e>] ? _raw_spin_lock_irqsave+0x2e/0x40
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717322] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717325] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717327] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717330] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717333] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717337] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717340] [<ffffffff810a152c>] ? do_futex+0x7c/0x1b0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717342] [<ffffffff810a17a2>] ? sys_futex+0x142/0x1a0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717345] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717348] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717349] Mem-Info:
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717351] Node 0 DMA per-cpu:
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717352] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717354] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717355] Node 0 DMA32 per-cpu:
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717357] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717358] CPU 1: hi: 186, btch: 31 usd: 37
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717360] active_anon:933122 inactive_anon:1393 isolated_anon:0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717361] active_file:22 inactive_file:38 isolated_file:0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717362] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717362] free:4024 slab_reclaimable:2445 slab_unreclaimable:2612
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717363] mapped:1352 shmem:1405 pagetables:3079 bounce:0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717365] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717371] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717374] Node 0 DMA32 free:7760kB min:7848kB low:9808kB high:11772kB active_anon:3732488kB inactive_anon:5572kB active_file:88kB inactive_file:152kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9780kB slab_unreclaimable:10448kB kernel_stack:728kB pagetables:12316kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:413 all_unreclaimable? yes
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717381] lowmem_reserve[]: 0 0 0 0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717383] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717390] Node 0 DMA32: 872*4kB 220*8kB 23*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7760kB
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717397] 1464 total pagecache pages
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717398] 0 pages in swap cache
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717399] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717400] Free swap = 0kB
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.717401] Total swap = 0kB
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722777] 985072 pages RAM
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722780] 24998 pages reserved
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722781] 3782 pages shared
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722782] 953312 pages non-shared
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722784] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722793] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722796] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722800] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722803] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722806] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722809] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722812] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722816] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722819] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722823] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722826] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722829] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722832] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722835] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722838] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722841] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722844] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722848] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722851] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722854] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722857] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722860] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722863] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722867] [13468] 0 13468 89584 2492 1 0 0 php5-fpm
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722870] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722873] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722876] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722879] [17508] 108 17508 6792 76 1 0 0 pickup
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722883] [22075] 1012 22075 874204 811211 0 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722886] [27317] 1012 27317 68580 5604 1 0 0 php
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722888] Out of memory: Kill process 22075 (php) score 846 or sacrifice child
Jul 15 22:26:33 ip-10-61-174-46 kernel: [1454088.722899] Killed process 22075 (php) total-vm:3496816kB, anon-rss:3244844kB, file-rss:0kB
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228142] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228146] php cpuset=/ mems_allowed=0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228149] Pid: 27565, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228151] Call Trace:
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228169] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228172] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228175] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228183] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228186] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228189] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228191] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228194] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228196] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228199] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228204] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228206] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228208] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228211] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228213] Mem-Info:
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228214] Node 0 DMA per-cpu:
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228216] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228217] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228218] Node 0 DMA32 per-cpu:
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228220] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228221] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228224] active_anon:933087 inactive_anon:1393 isolated_anon:0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228224] active_file:10 inactive_file:18 isolated_file:0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228225] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228226] free:4048 slab_reclaimable:2536 slab_unreclaimable:2614
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228226] mapped:1333 shmem:1405 pagetables:3072 bounce:0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228234] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228237] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732348kB inactive_anon:5572kB active_file:40kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5332kB shmem:5620kB slab_reclaimable:10144kB slab_unreclaimable:10456kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:19603 all_unreclaimable? yes
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228244] lowmem_reserve[]: 0 0 0 0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228246] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228253] Node 0 DMA32: 916*4kB 267*8kB 8*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8008kB
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228259] 1418 total pagecache pages
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228261] 0 pages in swap cache
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228262] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228263] Free swap = 0kB
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.228264] Total swap = 0kB
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233452] 985072 pages RAM
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233455] 24998 pages reserved
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233456] 3778 pages shared
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233457] 953321 pages non-shared
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233459] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233468] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233471] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233474] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233478] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233481] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233485] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233488] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233491] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233495] [ 790] 101 790 63477 639 0 0 0 rsyslogd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233498] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233502] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233505] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233508] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233511] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233514] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233517] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233520] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233524] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233527] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233530] [ 654] 1012 654 70351 5764 0 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233533] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233536] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233539] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233542] [13468] 0 13468 89584 2510 0 0 0 php5-fpm
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233546] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233549] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233552] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233555] [17508] 108 17508 6792 76 0 0 0 pickup
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233558] [27565] 1012 27565 874213 811151 1 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233562] [30511] 1012 30511 68580 5605 1 0 0 php
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233564] Out of memory: Kill process 27565 (php) score 846 or sacrifice child
Jul 15 22:30:38 ip-10-61-174-46 kernel: [1454334.233576] Killed process 27565 (php) total-vm:3496852kB, anon-rss:3244604kB, file-rss:0kB
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900125] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900129] php cpuset=/ mems_allowed=0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900131] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900133] Call Trace:
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900140] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900150] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900154] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900156] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900159] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900163] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900166] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900168] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900171] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900174] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900176] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900179] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900181] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900184] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900186] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900188] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900190] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900193] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900194] Mem-Info:
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900195] Node 0 DMA per-cpu:
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900197] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900199] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900200] Node 0 DMA32 per-cpu:
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900201] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900203] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900205] active_anon:933069 inactive_anon:1393 isolated_anon:0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900206] active_file:9 inactive_file:13 isolated_file:0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900206] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900207] free:4048 slab_reclaimable:2614 slab_unreclaimable:2612
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900208] mapped:1356 shmem:1405 pagetables:2987 bounce:0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900209] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900216] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900219] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732276kB inactive_anon:5572kB active_file:36kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10456kB slab_unreclaimable:10448kB kernel_stack:720kB pagetables:11948kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14750 all_unreclaimable? yes
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900225] lowmem_reserve[]: 0 0 0 0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900228] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900235] Node 0 DMA32: 1461*4kB 13*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8028kB
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900241] 1479 total pagecache pages
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900242] 0 pages in swap cache
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900244] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900245] Free swap = 0kB
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.900246] Total swap = 0kB
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905380] 985072 pages RAM
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905383] 24998 pages reserved
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905384] 3812 pages shared
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905385] 953285 pages non-shared
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905387] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905395] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905399] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905402] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905405] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905409] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905412] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905415] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905418] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905423] [ 790] 101 790 63477 640 0 0 0 rsyslogd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905426] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905429] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905433] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905436] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905439] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905442] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905445] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905448] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905451] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905455] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905457] [ 654] 1012 654 70351 5774 1 0 0 php
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905461] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905464] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905467] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905470] [13468] 0 13468 89584 2528 0 0 0 php5-fpm
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905473] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905476] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905479] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905482] [30695] 1012 30695 879848 816733 0 0 0 php
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905486] [ 7610] 108 7610 6792 71 0 0 0 pickup
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905488] Out of memory: Kill process 30695 (php) score 852 or sacrifice child
Jul 15 22:47:47 ip-10-61-174-46 kernel: [1455362.905499] Killed process 30695 (php) total-vm:3519392kB, anon-rss:3266932kB, file-rss:0kB
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716321] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716325] php cpuset=/ mems_allowed=0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716328] Pid: 18235, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716329] Call Trace:
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716336] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716339] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716341] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716344] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716347] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716351] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716353] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716356] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716360] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716363] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716365] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716368] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716371] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716373] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716376] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716378] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716380] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716382] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716384] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716387] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716389] Mem-Info:
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716390] Node 0 DMA per-cpu:
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716392] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716393] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716394] Node 0 DMA32 per-cpu:
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716396] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716397] CPU 1: hi: 186, btch: 31 usd: 59
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716400] active_anon:933012 inactive_anon:1393 isolated_anon:0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716400] active_file:17 inactive_file:0 isolated_file:0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716401] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716402] free:4043 slab_reclaimable:2420 slab_unreclaimable:2641
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716402] mapped:1341 shmem:1405 pagetables:3138 bounce:0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716404] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716410] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716413] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3732048kB inactive_anon:5572kB active_file:68kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5364kB shmem:5620kB slab_reclaimable:9680kB slab_unreclaimable:10564kB kernel_stack:760kB pagetables:12552kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:219 all_unreclaimable? yes
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716420] lowmem_reserve[]: 0 0 0 0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716422] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716429] Node 0 DMA32: 842*4kB 209*8kB 41*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716435] 1458 total pagecache pages
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716437] 0 pages in swap cache
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716438] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716439] Free swap = 0kB
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.716440] Total swap = 0kB
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721640] 985072 pages RAM
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721643] 24998 pages reserved
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721644] 3780 pages shared
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721645] 953268 pages non-shared
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721647] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721656] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721659] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721662] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721665] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721669] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721672] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721675] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721679] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721683] [ 790] 101 790 63477 640 0 0 0 rsyslogd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721686] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721689] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721692] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721695] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721698] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721701] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721705] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721708] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721711] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721714] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721717] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721720] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721723] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721727] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721730] [13468] 0 13468 89584 2513 0 0 0 php5-fpm
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721733] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721736] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721739] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721743] [ 7610] 108 7610 6792 77 1 0 0 pickup
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721746] [11079] 1012 11079 873675 810601 1 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721749] [17976] 108 17976 6818 74 1 0 0 cleanup
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721752] [17977] 108 17977 6795 75 0 0 0 trivial-rewrite
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721755] [17978] 108 17978 11107 158 0 0 0 smtp
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721758] [17982] 108 17982 11107 159 1 0 0 smtp
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721761] [18235] 1012 18235 68580 5604 0 0 0 php
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721764] Out of memory: Kill process 11079 (php) score 846 or sacrifice child
Jul 15 22:57:30 ip-10-61-174-46 kernel: [1455945.721775] Killed process 11079 (php) total-vm:3494700kB, anon-rss:3242404kB, file-rss:0kB
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968226] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968231] php cpuset=/ mems_allowed=0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968234] Pid: 18590, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968237] Call Trace:
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968244] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968248] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968251] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968254] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968260] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968264] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968269] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968272] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968275] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968279] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968283] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968287] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968290] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968293] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968297] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968300] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968304] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968307] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968309] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968312] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968314] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968316] Mem-Info:
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968317] Node 0 DMA per-cpu:
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968319] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968320] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968321] Node 0 DMA32 per-cpu:
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968323] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968324] CPU 1: hi: 186, btch: 31 usd: 25
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968327] active_anon:932619 inactive_anon:1393 isolated_anon:0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968328] active_file:0 inactive_file:0 isolated_file:0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968328] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968329] free:4048 slab_reclaimable:2708 slab_unreclaimable:2643
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968330] mapped:1368 shmem:1405 pagetables:3233 bounce:0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968331] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968338] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968341] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730476kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:10832kB slab_unreclaimable:10572kB kernel_stack:776kB pagetables:12932kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:26 all_unreclaimable? yes
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968347] lowmem_reserve[]: 0 0 0 0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968349] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968356] Node 0 DMA32: 789*4kB 182*8kB 42*16kB 9*32kB 5*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968363] 1437 total pagecache pages
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968364] 0 pages in swap cache
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968365] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968366] Free swap = 0kB
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.968367] Total swap = 0kB
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973576] 985072 pages RAM
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973578] 24998 pages reserved
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973579] 3779 pages shared
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973580] 953307 pages non-shared
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973582] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973591] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973594] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973598] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973601] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973604] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973607] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973611] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973614] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973618] [ 790] 101 790 63477 640 1 0 0 rsyslogd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973621] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973624] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973628] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973631] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973634] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973637] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973640] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973643] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973646] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973649] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973653] [ 654] 1012 654 70351 5764 0 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973656] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973659] [ 658] 1012 658 152314 88755 1 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973662] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973665] [13468] 0 13468 89584 2504 0 0 0 php5-fpm
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973668] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973671] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973674] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973678] [ 7610] 108 7610 6792 77 0 0 0 pickup
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973681] [17976] 108 17976 6818 82 0 0 0 cleanup
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973684] [17977] 108 17977 6795 75 0 0 0 trivial-rewrite
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973688] [17978] 108 17978 11107 159 1 0 0 smtp
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973690] [18572] 108 18572 11107 158 0 0 0 smtp
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973694] [18590] 1012 18590 872880 809834 0 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973697] [19851] 108 19851 11107 159 0 0 0 smtp
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973700] [19859] 108 19859 11108 159 0 0 0 smtp
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973703] [19860] 108 19860 6792 69 1 0 0 scache
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973706] [21291] 1012 21291 68580 5604 1 0 0 php
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973709] Out of memory: Kill process 18590 (php) score 845 or sacrifice child
Jul 15 23:01:29 ip-10-61-174-46 kernel: [1456184.973720] Killed process 18590 (php) total-vm:3491520kB, anon-rss:3239336kB, file-rss:0kB
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923326] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923331] php cpuset=/ mems_allowed=0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923335] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923337] Call Trace:
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923345] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923349] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923351] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923354] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923357] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923361] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923363] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923366] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923369] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923372] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923374] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923377] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923379] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923383] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923387] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923390] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923393] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923396] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923399] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923401] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923404] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923406] Mem-Info:
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923407] Node 0 DMA per-cpu:
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923409] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923410] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923412] Node 0 DMA32 per-cpu:
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923413] CPU 0: hi: 186, btch: 31 usd: 35
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923415] CPU 1: hi: 186, btch: 31 usd: 66
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923417] active_anon:932931 inactive_anon:1393 isolated_anon:0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923418] active_file:42 inactive_file:12 isolated_file:17
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923419] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923419] free:3997 slab_reclaimable:2440 slab_unreclaimable:2617
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923420] mapped:1346 shmem:1405 pagetables:3211 bounce:0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923422] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923428] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923431] Node 0 DMA32 free:7652kB min:7848kB low:9808kB high:11772kB active_anon:3731724kB inactive_anon:5572kB active_file:168kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):68kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5384kB shmem:5620kB slab_reclaimable:9760kB slab_unreclaimable:10468kB kernel_stack:776kB pagetables:12844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:244 all_unreclaimable? yes
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923438] lowmem_reserve[]: 0 0 0 0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923440] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923447] Node 0 DMA32: 865*4kB 216*8kB 24*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7652kB
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923454] 1438 total pagecache pages
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923455] 0 pages in swap cache
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923456] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923458] Free swap = 0kB
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.923458] Total swap = 0kB
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928554] 985072 pages RAM
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928556] 24998 pages reserved
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928557] 3786 pages shared
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928559] 953295 pages non-shared
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928561] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928569] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928573] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928576] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928579] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928582] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928586] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928589] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928593] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928596] [ 790] 101 790 63477 640 1 0 0 rsyslogd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928599] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928603] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928606] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928609] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928612] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928615] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928618] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928622] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928625] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928628] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928631] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928634] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928637] [ 658] 1012 658 152314 88756 1 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928640] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928644] [13468] 0 13468 89584 2528 1 0 0 php5-fpm
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928647] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928649] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928653] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928656] [ 7610] 108 7610 6792 77 1 0 0 pickup
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928660] [17976] 108 17976 6818 82 0 0 0 cleanup
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928663] [17977] 108 17977 6795 75 1 0 0 trivial-rewrite
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928666] [17978] 108 17978 11107 159 1 0 0 smtp
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928669] [18572] 108 18572 11107 158 1 0 0 smtp
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928672] [19860] 108 19860 6792 69 0 0 0 scache
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928675] [21574] 1012 21574 867673 804644 0 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928678] [22499] 1012 22499 74197 11243 1 0 0 php
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928681] [22576] 108 22576 11107 159 1 0 0 smtp
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928684] Out of memory: Kill process 21574 (php) score 839 or sacrifice child
Jul 15 23:03:03 ip-10-61-174-46 kernel: [1456278.928694] Killed process 21574 (php) total-vm:3470692kB, anon-rss:3218576kB, file-rss:0kB
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945151] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945155] php cpuset=/ mems_allowed=0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945158] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945159] Call Trace:
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945167] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945171] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945174] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945177] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945181] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945183] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945186] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945189] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945192] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945197] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945202] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945206] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945209] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945212] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945214] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945217] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945219] Mem-Info:
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945221] Node 0 DMA per-cpu:
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945222] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945225] Node 0 DMA32 per-cpu:
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945227] CPU 0: hi: 186, btch: 31 usd: 61
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945228] CPU 1: hi: 186, btch: 31 usd: 14
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945231] active_anon:932569 inactive_anon:1393 isolated_anon:0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945231] active_file:23 inactive_file:8 isolated_file:12
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945232] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945233] free:4040 slab_reclaimable:2706 slab_unreclaimable:2652
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945233] mapped:1363 shmem:1405 pagetables:3232 bounce:0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945241] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945244] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3730276kB inactive_anon:5572kB active_file:92kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:10824kB slab_unreclaimable:10608kB kernel_stack:768kB pagetables:12928kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:324 all_unreclaimable? yes
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945251] lowmem_reserve[]: 0 0 0 0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945260] Node 0 DMA32: 1326*4kB 59*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945267] 1467 total pagecache pages
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945268] 0 pages in swap cache
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945269] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945270] Free swap = 0kB
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.945271] Total swap = 0kB
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950388] 985072 pages RAM
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950391] 24998 pages reserved
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950392] 3786 pages shared
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950393] 953279 pages non-shared
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950395] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950403] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950407] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950410] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950413] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950416] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950420] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950423] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950426] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950430] [ 790] 101 790 63477 640 1 0 0 rsyslogd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950433] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950436] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950440] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950443] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950446] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950449] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950453] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950456] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950459] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950462] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950465] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950468] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950471] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950474] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950478] [13468] 0 13468 89584 2513 1 0 0 php5-fpm
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950481] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950484] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950487] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950491] [ 7610] 108 7610 6792 77 0 0 0 pickup
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950494] [22808] 1012 22808 820406 757310 0 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950497] [32582] 108 32582 6818 73 1 0 0 cleanup
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950500] [32583] 108 32583 6795 70 1 0 0 trivial-rewrite
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950503] [32588] 108 32588 11107 158 1 0 0 smtp
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950507] [ 3709] 1012 3709 68580 5604 1 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950510] [ 3870] 1012 3870 1099 25 1 0 0 sh
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950513] [ 3871] 1012 3871 68695 5741 0 0 0 php
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950516] Out of memory: Kill process 22808 (php) score 790 or sacrifice child
Jul 15 23:21:29 ip-10-61-174-46 kernel: [1457384.950527] Killed process 22808 (php) total-vm:3281624kB, anon-rss:3029240kB, file-rss:0kB
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464153] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464157] php cpuset=/ mems_allowed=0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464159] Pid: 3950, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464161] Call Trace:
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464168] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464170] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464173] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464175] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464181] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464184] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464187] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464189] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464192] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464195] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464197] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464201] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464203] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464205] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464211] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464213] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464216] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464220] Mem-Info:
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464222] Node 0 DMA per-cpu:
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464223] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464225] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464226] Node 0 DMA32 per-cpu:
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464227] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464229] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464231] active_anon:933040 inactive_anon:1393 isolated_anon:0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464232] active_file:9 inactive_file:10 isolated_file:0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464233] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464233] free:4048 slab_reclaimable:2419 slab_unreclaimable:2683
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464234] mapped:1368 shmem:1405 pagetables:3111 bounce:0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464236] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464242] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464245] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732160kB inactive_anon:5572kB active_file:36kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:9676kB slab_unreclaimable:10732kB kernel_stack:744kB pagetables:12444kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:336 all_unreclaimable? yes
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464251] lowmem_reserve[]: 0 0 0 0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464254] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464261] Node 0 DMA32: 989*4kB 217*8kB 7*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464267] 1446 total pagecache pages
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464268] 0 pages in swap cache
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464270] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464271] Free swap = 0kB
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.464272] Total swap = 0kB
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469470] 985072 pages RAM
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469472] 24998 pages reserved
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469473] 3806 pages shared
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469474] 953306 pages non-shared
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469476] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469484] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469488] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469491] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469494] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469498] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469501] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469504] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469508] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469512] [ 790] 101 790 63477 640 0 0 0 rsyslogd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469515] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469518] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469522] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469525] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469528] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469531] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469534] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469537] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469541] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469544] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469547] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469550] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469553] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469556] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469559] [13468] 0 13468 89584 2537 0 0 0 php5-fpm
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469563] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469565] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469569] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469572] [32583] 108 32583 6795 74 1 0 0 trivial-rewrite
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469576] [ 3950] 1012 3950 826788 763738 0 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469579] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469582] [ 4376] 108 4376 6792 70 1 0 0 scache
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469585] [ 5950] 1012 5950 68580 5603 1 0 0 php
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469588] Out of memory: Kill process 3950 (php) score 797 or sacrifice child
Jul 15 23:24:39 ip-10-61-174-46 kernel: [1457575.469600] Killed process 3950 (php) total-vm:3307152kB, anon-rss:3054908kB, file-rss:44kB
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492137] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492142] php cpuset=/ mems_allowed=0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492146] Pid: 6159, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492148] Call Trace:
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492155] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492159] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492168] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492171] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492174] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492177] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492180] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492182] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492185] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492187] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492191] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492193] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492195] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492198] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492201] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492202] Mem-Info:
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492204] Node 0 DMA per-cpu:
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492206] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492207] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492208] Node 0 DMA32 per-cpu:
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492210] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492211] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492213] active_anon:932122 inactive_anon:1393 isolated_anon:0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492214] active_file:15 inactive_file:17 isolated_file:0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492215] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492215] free:4048 slab_reclaimable:2692 slab_unreclaimable:2714
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492216] mapped:1383 shmem:1405 pagetables:3644 bounce:0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492218] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492224] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492227] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728488kB inactive_anon:5572kB active_file:60kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5532kB shmem:5620kB slab_reclaimable:10768kB slab_unreclaimable:10856kB kernel_stack:840kB pagetables:14576kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:16629 all_unreclaimable? yes
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492234] lowmem_reserve[]: 0 0 0 0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492236] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492243] Node 0 DMA32: 882*4kB 261*8kB 14*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7920kB
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492249] 1452 total pagecache pages
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492251] 0 pages in swap cache
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492252] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492253] Free swap = 0kB
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.492254] Total swap = 0kB
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497640] 985072 pages RAM
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497642] 24998 pages reserved
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497643] 3781 pages shared
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497645] 953322 pages non-shared
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497646] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497655] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497659] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497662] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497665] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497668] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497672] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497675] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497678] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497682] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497685] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497689] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497692] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497695] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497698] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497701] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497704] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497707] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497710] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497714] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497717] [ 654] 1012 654 70351 5765 0 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497720] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497723] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497726] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497729] [13468] 0 13468 89584 2522 0 0 0 php5-fpm
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497732] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497735] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497739] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497742] [ 4355] 108 4355 6792 76 0 0 0 pickup
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497745] [ 4376] 108 4376 6792 70 1 0 0 scache
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497748] [ 6159] 1012 6159 802148 739098 0 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497751] [ 6379] 108 6379 6818 73 0 0 0 cleanup
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497754] [ 6380] 108 6380 6795 70 0 0 0 trivial-rewrite
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497757] [ 6381] 108 6381 11107 159 1 0 0 smtp
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497760] [ 6630] 1012 6630 68580 5604 1 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497763] [ 6722] 1012 6722 1099 25 0 0 0 sh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497766] [ 6723] 1012 6723 1099 24 0 0 0 sh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497769] [ 6724] 1012 6724 68825 5873 1 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497772] [ 6725] 1012 6725 68825 5874 1 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497775] [ 6736] 1012 6736 1099 26 0 0 0 sh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497778] [ 6737] 1012 6737 14040 63 0 0 0 git
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497781] [ 6738] 1012 6738 1099 26 0 0 0 sh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497784] [ 6739] 1012 6739 68499 5552 0 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497787] [ 6740] 1012 6740 4658 58 0 0 0 git
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497790] [ 6741] 1012 6741 68499 5552 1 0 0 php
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497793] [ 6744] 1012 6744 1099 25 1 0 0 sh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497796] [ 6745] 1012 6745 10901 165 1 0 0 ssh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497799] [ 6746] 1012 6746 1099 24 1 0 0 sh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497802] [ 6747] 1012 6747 10901 164 1 0 0 ssh
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497805] Out of memory: Kill process 6159 (php) score 771 or sacrifice child
Jul 15 23:25:47 ip-10-61-174-46 kernel: [1457643.497816] Killed process 6159 (php) total-vm:3208592kB, anon-rss:2956368kB, file-rss:24kB
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984539] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984543] php cpuset=/ mems_allowed=0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984545] Pid: 6815, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984547] Call Trace:
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984554] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984556] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984558] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984561] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984564] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984568] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984570] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984573] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984576] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984578] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984582] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984584] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984586] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984590] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984592] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984594] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984597] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984600] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984601] Mem-Info:
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984603] Node 0 DMA per-cpu:
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984604] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984606] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984607] Node 0 DMA32 per-cpu:
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984609] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984610] CPU 1: hi: 186, btch: 31 usd: 25
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984613] active_anon:932871 inactive_anon:1393 isolated_anon:0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984613] active_file:0 inactive_file:0 isolated_file:10
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984614] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984615] free:4034 slab_reclaimable:2634 slab_unreclaimable:2615
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984615] mapped:1360 shmem:1405 pagetables:3133 bounce:0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984617] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984623] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984626] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3731484kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):40kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:10536kB slab_unreclaimable:10460kB kernel_stack:752kB pagetables:12532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:37 all_unreclaimable? yes
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984632] lowmem_reserve[]: 0 0 0 0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984635] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984642] Node 0 DMA32: 936*4kB 211*8kB 20*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984648] 1420 total pagecache pages
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984650] 0 pages in swap cache
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984651] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984652] Free swap = 0kB
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.984653] Total swap = 0kB
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990027] 985072 pages RAM
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990030] 24998 pages reserved
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990031] 3780 pages shared
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990032] 953313 pages non-shared
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990034] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990042] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990046] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990049] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990052] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990056] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990059] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990062] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990065] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990069] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990073] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990076] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990079] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990082] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990085] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990089] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990092] [ 1051] 0 1051 201628 844 1 0 0 collectd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990095] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990098] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990101] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990104] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990108] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990111] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990113] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990117] [13468] 0 13468 89584 2507 1 0 0 php5-fpm
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990120] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990123] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990126] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990130] [ 4355] 108 4355 6792 76 1 0 0 pickup
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990133] [ 6815] 1012 6815 826402 763380 0 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990135] [ 9560] 108 9560 6818 74 1 0 0 cleanup
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990139] [ 9561] 108 9561 6795 70 1 0 0 trivial-rewrite
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990142] [ 9562] 108 9562 11107 159 0 0 0 smtp
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990145] [ 9720] 1012 9720 68580 5605 1 0 0 php
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990147] Out of memory: Kill process 6815 (php) score 796 or sacrifice child
Jul 15 23:30:13 ip-10-61-174-46 kernel: [1457908.990159] Killed process 6815 (php) total-vm:3305608kB, anon-rss:3053516kB, file-rss:4kB
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004125] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004130] php5-fpm cpuset=/ mems_allowed=0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004133] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004136] Call Trace:
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004144] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004147] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004154] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004160] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004163] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004168] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004171] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004174] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004179] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004182] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004186] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004189] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004196] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004200] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004205] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004209] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004212] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004215] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004219] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004222] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004224] Mem-Info:
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004225] Node 0 DMA per-cpu:
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004227] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004229] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004230] Node 0 DMA32 per-cpu:
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004231] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004233] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004235] active_anon:932692 inactive_anon:1393 isolated_anon:0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004236] active_file:10 inactive_file:14 isolated_file:0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004237] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004237] free:4048 slab_reclaimable:2588 slab_unreclaimable:2620
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004238] mapped:1366 shmem:1405 pagetables:3298 bounce:0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004239] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004246] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004249] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730768kB inactive_anon:5572kB active_file:40kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:10352kB slab_unreclaimable:10480kB kernel_stack:768kB pagetables:13192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1526 all_unreclaimable? yes
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004255] lowmem_reserve[]: 0 0 0 0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004258] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004265] Node 0 DMA32: 1120*4kB 185*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8024kB
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004271] 1442 total pagecache pages
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004273] 0 pages in swap cache
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004274] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004275] Free swap = 0kB
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.004276] Total swap = 0kB
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009427] 985072 pages RAM
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009429] 24998 pages reserved
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009430] 3789 pages shared
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009432] 953303 pages non-shared
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009433] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009442] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009446] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009449] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009452] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009455] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009458] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009462] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009465] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009469] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009472] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009475] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009478] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009481] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009485] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009488] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009491] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009494] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009497] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009500] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009503] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009506] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009509] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009512] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009516] [13468] 0 13468 89584 2528 0 0 0 php5-fpm
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009518] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009521] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009525] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009528] [ 4355] 108 4355 6792 76 1 0 0 pickup
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009532] [10608] 108 10608 6792 70 1 0 0 scache
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009535] [11530] 1012 11530 814983 751968 0 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009538] [12317] 1012 12317 68580 5604 1 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009540] [12453] 1012 12453 1099 24 0 0 0 sh
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009544] [12454] 1012 12454 68825 5871 1 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009547] [12460] 1012 12460 1099 25 0 0 0 sh
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009550] [12461] 1012 12461 4776 59 0 0 0 git
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009553] [12462] 1012 12462 68434 5488 1 0 0 php
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009556] Out of memory: Kill process 11530 (php) score 784 or sacrifice child
Jul 15 23:33:42 ip-10-61-174-46 kernel: [1458118.009567] Killed process 11530 (php) total-vm:3259932kB, anon-rss:3007872kB, file-rss:0kB
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048143] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048147] php cpuset=/ mems_allowed=0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048149] Pid: 12559, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048151] Call Trace:
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048158] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048171] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048174] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048177] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048182] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048186] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048189] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048192] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048196] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048199] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048201] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048204] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048206] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048211] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048213] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048215] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048218] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048220] Mem-Info:
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048221] Node 0 DMA per-cpu:
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048223] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048225] Node 0 DMA32 per-cpu:
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048227] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048228] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048230] active_anon:932972 inactive_anon:1393 isolated_anon:0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048231] active_file:0 inactive_file:0 isolated_file:0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048232] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048232] free:4048 slab_reclaimable:2592 slab_unreclaimable:2596
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048233] mapped:1359 shmem:1405 pagetables:3095 bounce:0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048241] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048244] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731888kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:10368kB slab_unreclaimable:10384kB kernel_stack:736kB pagetables:12380kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048250] lowmem_reserve[]: 0 0 0 0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048260] Node 0 DMA32: 868*4kB 216*8kB 34*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048266] 1471 total pagecache pages
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048267] 0 pages in swap cache
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048269] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048270] Free swap = 0kB
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.048271] Total swap = 0kB
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053463] 985072 pages RAM
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053466] 24998 pages reserved
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053467] 3789 pages shared
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053468] 953348 pages non-shared
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053470] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053478] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053481] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053484] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053487] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053491] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053494] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053497] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053500] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053505] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053508] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053511] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053514] [ 975] 0 975 6276 80 1 0 0 master
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053517] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053521] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053524] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053527] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053530] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053533] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053536] [15657] 0 15657 4777 60 1 0 0 cron
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053539] [ 654] 1012 654 70351 5764 1 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053543] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053546] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053549] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053552] [13468] 0 13468 89584 2513 1 0 0 php5-fpm
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053555] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053558] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053561] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053565] [ 4355] 108 4355 6792 76 0 0 0 pickup
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053568] [12559] 1012 12559 826791 763694 1 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053571] [15357] 108 15357 6792 69 0 0 0 scache
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053574] [16545] 1012 16545 68582 5632 0 0 0 php
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053577] Out of memory: Kill process 12559 (php) score 797 or sacrifice child
Jul 15 23:39:45 ip-10-61-174-46 kernel: [1458481.053588] Killed process 12559 (php) total-vm:3307164kB, anon-rss:3054776kB, file-rss:0kB
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088719] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088723] php cpuset=/ mems_allowed=0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088727] Pid: 16727, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088728] Call Trace:
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088735] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088738] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088740] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088743] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088746] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088749] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088752] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088755] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088757] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088761] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088763] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088765] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088769] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088771] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088773] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088776] [<ffffffff8165c2f1>] ? __schedule+0x6b1/0x6f0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088779] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088781] Mem-Info:
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088782] Node 0 DMA per-cpu:
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088784] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088785] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088786] Node 0 DMA32 per-cpu:
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088788] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088789] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088792] active_anon:932678 inactive_anon:1393 isolated_anon:0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088792] active_file:30 inactive_file:31 isolated_file:0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088793] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088794] free:4048 slab_reclaimable:2565 slab_unreclaimable:2650
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088794] mapped:1377 shmem:1405 pagetables:3300 bounce:0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088796] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088803] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088805] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730712kB inactive_anon:5572kB active_file:120kB inactive_file:124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:10260kB slab_unreclaimable:10600kB kernel_stack:784kB pagetables:13200kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:367 all_unreclaimable? yes
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088812] lowmem_reserve[]: 0 0 0 0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088815] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088821] Node 0 DMA32: 717*4kB 177*8kB 67*16kB 17*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088828] 1494 total pagecache pages
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088829] 0 pages in swap cache
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088831] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088832] Free swap = 0kB
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.088833] Total swap = 0kB
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094213] 985072 pages RAM
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094215] 24998 pages reserved
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094217] 3810 pages shared
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094218] 953305 pages non-shared
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094219] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094228] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094232] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094235] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094238] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094241] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094244] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094248] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094251] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094255] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094258] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094261] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094265] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094268] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094271] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094274] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094277] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094280] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094283] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094286] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094289] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094292] [ 657] 1012 657 69242 6259 0 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094295] [ 658] 1012 658 200573 135993 0 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094298] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094302] [13468] 0 13468 89584 2498 0 0 0 php5-fpm
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094305] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094308] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094311] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094315] [ 4355] 108 4355 6792 76 0 0 0 pickup
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094318] [16727] 1012 16727 814788 751731 0 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094321] [18743] 1012 18743 68580 5605 1 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094324] [18952] 1012 18952 1099 25 0 0 0 sh
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094326] [18953] 1012 18953 68825 5871 0 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094329] [18959] 1012 18959 1099 25 0 0 0 sh
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094332] [18960] 1012 18960 4700 70 0 0 0 git
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094335] [18961] 1012 18961 68499 5551 0 0 0 php
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094338] [18963] 1012 18963 1099 24 0 0 0 sh
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094341] [18964] 1012 18964 10901 164 1 0 0 ssh
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094344] Out of memory: Kill process 16727 (php) score 784 or sacrifice child
Jul 15 23:43:10 ip-10-61-174-46 kernel: [1458686.094354] Killed process 16727 (php) total-vm:3259152kB, anon-rss:3006848kB, file-rss:76kB
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960138] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960142] collectd cpuset=/ mems_allowed=0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960144] Pid: 1077, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960146] Call Trace:
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960155] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960158] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960160] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960164] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960167] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960169] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960172] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960174] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960178] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960180] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960183] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960186] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960190] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960193] [<ffffffff810a152c>] ? do_futex+0x7c/0x1b0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960195] [<ffffffff810a17a2>] ? sys_futex+0x142/0x1a0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960198] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960201] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960203] Mem-Info:
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960204] Node 0 DMA per-cpu:
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960206] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960207] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960208] Node 0 DMA32 per-cpu:
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960210] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960211] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960214] active_anon:932327 inactive_anon:1393 isolated_anon:0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960215] active_file:14 inactive_file:14 isolated_file:0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960215] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960216] free:4048 slab_reclaimable:2864 slab_unreclaimable:2679
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960217] mapped:1365 shmem:1405 pagetables:3302 bounce:0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960218] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960225] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960228] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729308kB inactive_anon:5572kB active_file:56kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:11456kB slab_unreclaimable:10716kB kernel_stack:784kB pagetables:13208kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:36303 all_unreclaimable? yes
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960234] lowmem_reserve[]: 0 0 0 0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960237] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960244] Node 0 DMA32: 1480*4kB 7*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8024kB
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960250] 1456 total pagecache pages
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960251] 0 pages in swap cache
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960253] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960254] Free swap = 0kB
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.960255] Total swap = 0kB
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965508] 985072 pages RAM
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965510] 24998 pages reserved
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965512] 3805 pages shared
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965513] 953283 pages non-shared
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965515] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965523] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965527] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965530] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965533] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965536] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965539] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965543] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965546] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965550] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965553] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965556] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965559] [ 975] 0 975 6276 80 0 0 0 master
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965563] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965566] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965569] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965572] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965575] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965578] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965582] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965585] [ 654] 1012 654 70351 5765 0 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965588] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965591] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965594] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965597] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965600] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965603] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965606] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965610] [ 4355] 108 4355 6792 76 0 0 0 pickup
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965613] [23845] 1012 23845 80111 17176 1 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965616] [23882] 108 23882 6795 71 1 0 0 trivial-rewrite
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965619] [25127] 108 25127 6818 73 1 0 0 cleanup
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965622] [25128] 108 25128 11107 158 1 0 0 smtp
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965625] [25138] 108 25138 11107 159 1 0 0 smtp
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965628] [25149] 108 25149 11107 158 1 0 0 smtp
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965631] [25155] 108 25155 11107 158 1 0 0 smtp
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965634] [25225] 1012 25225 808264 745205 0 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965637] [25434] 1012 25434 68580 5605 1 0 0 php
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965640] Out of memory: Kill process 25225 (php) score 777 or sacrifice child
Jul 15 23:52:33 ip-10-61-174-46 kernel: [1459248.965651] Killed process 25225 (php) total-vm:3233056kB, anon-rss:2980780kB, file-rss:40kB
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516146] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516150] php cpuset=/ mems_allowed=0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516152] Pid: 23845, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516154] Call Trace:
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516161] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516163] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516165] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516168] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516172] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516175] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516177] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516181] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516183] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516186] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516189] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516191] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516193] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516196] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516199] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516201] Mem-Info:
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516202] Node 0 DMA per-cpu:
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516204] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516205] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516206] Node 0 DMA32 per-cpu:
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516208] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516209] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516212] active_anon:932804 inactive_anon:1393 isolated_anon:0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516212] active_file:0 inactive_file:10 isolated_file:0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516213] unevictable:0 dirty:11 writeback:0 unstable:0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516214] free:4048 slab_reclaimable:2602 slab_unreclaimable:2632
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516214] mapped:1358 shmem:1405 pagetables:3198 bounce:0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516216] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516223] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731216kB inactive_anon:5572kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:44kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:10408kB slab_unreclaimable:10528kB kernel_stack:776kB pagetables:12792kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6768 all_unreclaimable? yes
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516232] lowmem_reserve[]: 0 0 0 0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516235] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516242] Node 0 DMA32: 839*4kB 209*8kB 44*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516248] 1472 total pagecache pages
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516249] 0 pages in swap cache
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516251] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516252] Free swap = 0kB
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.516253] Total swap = 0kB
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521397] 985072 pages RAM
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521399] 24998 pages reserved
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521400] 3819 pages shared
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521401] 953327 pages non-shared
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521403] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521412] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521415] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521419] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521422] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521425] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521428] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521431] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521435] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521439] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521442] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521445] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521448] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521451] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521454] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521458] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521461] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521464] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521467] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521470] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521473] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521476] [ 657] 1012 657 69242 6261 1 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521479] [ 658] 1012 658 200573 135994 1 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521482] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521485] [13468] 0 13468 89584 2517 1 0 0 php5-fpm
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521489] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521492] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521495] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521498] [ 4355] 108 4355 6792 76 0 0 0 pickup
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521502] [23845] 1012 23845 825933 762847 0 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521505] [23882] 108 23882 6795 75 1 0 0 trivial-rewrite
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521508] [25127] 108 25127 6818 81 0 0 0 cleanup
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521511] [25128] 108 25128 11107 157 0 0 0 smtp
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521514] [25138] 108 25138 11107 159 0 0 0 smtp
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521517] [25149] 108 25149 11107 157 1 0 0 smtp
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521520] [25155] 108 25155 11107 157 1 0 0 smtp
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521523] [26168] 1012 26168 68580 5608 1 0 0 php
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521526] Out of memory: Kill process 23845 (php) score 796 or sacrifice child
Jul 15 23:53:41 ip-10-61-174-46 kernel: [1459317.521536] Killed process 23845 (php) total-vm:3303732kB, anon-rss:3051360kB, file-rss:28kB
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720125] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720129] php cpuset=/ mems_allowed=0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720131] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720133] Call Trace:
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720140] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720150] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720154] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720156] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720159] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720162] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720164] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720172] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720176] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720179] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720182] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720186] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720188] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720191] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720193] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720196] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720198] Mem-Info:
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720199] Node 0 DMA per-cpu:
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720201] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720202] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720203] Node 0 DMA32 per-cpu:
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720205] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720206] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720209] active_anon:932636 inactive_anon:1393 isolated_anon:0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720209] active_file:0 inactive_file:0 isolated_file:0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720210] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720211] free:4047 slab_reclaimable:2582 slab_unreclaimable:2687
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720211] mapped:1356 shmem:1405 pagetables:3284 bounce:0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720213] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720219] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720222] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3730544kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10328kB slab_unreclaimable:10748kB kernel_stack:800kB pagetables:13136kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720229] lowmem_reserve[]: 0 0 0 0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720231] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720238] Node 0 DMA32: 827*4kB 139*8kB 31*16kB 23*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7892kB
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720245] 1520 total pagecache pages
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720246] 0 pages in swap cache
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720247] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720248] Free swap = 0kB
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.720249] Total swap = 0kB
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725513] 985072 pages RAM
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725515] 24998 pages reserved
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725516] 3815 pages shared
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725517] 953308 pages non-shared
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725519] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725528] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725532] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725535] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725538] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725541] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725545] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725548] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725552] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725556] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725559] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725562] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725565] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725569] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725572] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725575] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725578] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725581] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725584] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725588] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725591] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725594] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725597] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725600] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725603] [13468] 0 13468 89584 2502 1 0 0 php5-fpm
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725606] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725609] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725612] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725616] [23882] 108 23882 6795 75 0 0 0 trivial-rewrite
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725619] [25127] 108 25127 6818 81 0 0 0 cleanup
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725622] [25128] 108 25128 11107 157 0 0 0 smtp
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725625] [25138] 108 25138 11107 159 1 0 0 smtp
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725628] [25149] 108 25149 11107 157 0 0 0 smtp
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725631] [25155] 108 25155 11107 158 0 0 0 smtp
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725634] [26404] 1012 26404 825293 762301 0 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725637] [26567] 108 26567 11107 158 0 0 0 smtp
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725640] [26582] 108 26582 6792 69 1 0 0 scache
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725643] [26583] 108 26583 11108 158 1 0 0 smtp
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725647] [26676] 108 26676 6792 70 0 0 0 pickup
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725650] [26844] 1012 26844 68580 5604 1 0 0 php
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725652] Out of memory: Kill process 26404 (php) score 795 or sacrifice child
Jul 15 23:54:38 ip-10-61-174-46 kernel: [1459373.725663] Killed process 26404 (php) total-vm:3301172kB, anon-rss:3049204kB, file-rss:0kB
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856131] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856134] php cpuset=/ mems_allowed=0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856137] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856139] Call Trace:
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856146] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856148] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856153] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856158] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856161] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856164] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856167] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856170] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856174] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856176] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856180] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856182] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856184] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856187] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856189] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856192] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856194] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856196] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856199] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856201] Mem-Info:
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856202] Node 0 DMA per-cpu:
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856204] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856205] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856206] Node 0 DMA32 per-cpu:
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856208] CPU 0: hi: 186, btch: 31 usd: 0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856209] CPU 1: hi: 186, btch: 31 usd: 0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856212] active_anon:932813 inactive_anon:1393 isolated_anon:0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856213] active_file:9 inactive_file:14 isolated_file:0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856213] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856214] free:4048 slab_reclaimable:2476 slab_unreclaimable:2662
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856215] mapped:1357 shmem:1405 pagetables:3258 bounce:0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856216] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856223] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856226] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731252kB inactive_anon:5572kB active_file:36kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9904kB slab_unreclaimable:10648kB kernel_stack:792kB pagetables:13032kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10770 all_unreclaimable? yes
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856232] lowmem_reserve[]: 0 0 0 0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856235] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856241] Node 0 DMA32: 1284*4kB 95*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8008kB
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856248] 1434 total pagecache pages
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856249] 0 pages in swap cache
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856250] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856251] Free swap = 0kB
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.856252] Total swap = 0kB
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861351] 985072 pages RAM
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861353] 24998 pages reserved
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861354] 3781 pages shared
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861355] 953312 pages non-shared
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861357] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861366] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861370] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861373] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861376] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861379] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861382] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861385] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861389] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861393] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861396] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861399] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861403] [ 975] 0 975 6276 79 1 0 0 master
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861406] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861409] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861412] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861415] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861418] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861421] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861424] [15657] 0 15657 4777 59 1 0 0 cron
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861427] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861430] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861433] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861436] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861440] [13468] 0 13468 89584 2526 0 0 0 php5-fpm
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861443] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861446] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861449] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861453] [23882] 108 23882 6795 75 0 0 0 trivial-rewrite
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861456] [25127] 108 25127 6818 81 0 0 0 cleanup
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861459] [25128] 108 25128 11107 158 0 0 0 smtp
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861462] [25138] 108 25138 11107 159 0 0 0 smtp
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861465] [25149] 108 25149 11107 158 0 0 0 smtp
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861468] [26567] 108 26567 11107 157 0 0 0 smtp
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861471] [26582] 108 26582 6792 73 0 0 0 scache
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861474] [26583] 108 26583 11108 157 0 0 0 smtp
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861477] [26676] 108 26676 6792 76 1 0 0 pickup
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861480] [27103] 1012 27103 825677 762578 1 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861483] [27496] 1012 27496 68580 5604 0 0 0 php
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861486] Out of memory: Kill process 27103 (php) score 795 or sacrifice child
Jul 15 23:55:45 ip-10-61-174-46 kernel: [1459440.861497] Killed process 27103 (php) total-vm:3302708kB, anon-rss:3050312kB, file-rss:0kB
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280220] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280228] php cpuset=/ mems_allowed=0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280231] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280232] Call Trace:
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280239] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280242] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280244] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280247] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280252] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280255] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280258] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280260] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280263] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280266] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280269] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280271] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280273] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280276] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280278] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280281] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280284] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280287] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280290] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280295] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280297] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280300] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280303] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280304] Mem-Info:
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280305] Node 0 DMA per-cpu:
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280307] CPU 0: hi: 0, btch: 1 usd: 0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280309] CPU 1: hi: 0, btch: 1 usd: 0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280310] Node 0 DMA32 per-cpu:
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280311] CPU 0: hi: 186, btch: 31 usd: 30
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280313] CPU 1: hi: 186, btch: 31 usd: 30
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280315] active_anon:931988 inactive_anon:1393 isolated_anon:0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280316] active_file:0 inactive_file:0 isolated_file:0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280317] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280317] free:4012 slab_reclaimable:2496 slab_unreclaimable:2729
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280318] mapped:1366 shmem:1405 pagetables:3800 bounce:0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280320] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280326] lowmem_reserve[]: 0 3772 3772 3772
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280329] Node 0 DMA32 free:7712kB min:7848kB low:9808kB high:11772kB active_anon:3727952kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9984kB slab_unreclaimable:10916kB kernel_stack:928kB pagetables:15200kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24 all_unreclaimable? yes
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280336] lowmem_reserve[]: 0 0 0 0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280338] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280345] Node 0 DMA32: 804*4kB 114*8kB 28*16kB 16*32kB 5*64kB 2*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7712kB
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280351] 1449 total pagecache pages
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280353] 0 pages in swap cache
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280354] Swap cache stats: add 0, delete 0, find 0/0
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280355] Free swap = 0kB
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.280356] Total swap = 0kB
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285627] 985072 pages RAM
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285630] 24998 pages reserved
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285631] 3787 pages shared
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285632] 953301 pages non-shared
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285634] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285642] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285646] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285649] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285652] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285655] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285658] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285662] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285665] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285669] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285672] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285675] [ 831] 0 831 4226 41 0 0 0 atd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285679] [ 975] 0 975 6276 79 0 0 0 master
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285682] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285685] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285688] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285691] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285694] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285697] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285700] [15657] 0 15657 4777 59 0 0 0 cron
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285703] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285706] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285709] [ 658] 1012 658 200573 135991 0 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285712] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285716] [13468] 0 13468 89584 2511 0 0 0 php5-fpm
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285719] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285722] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285725] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285729] [23882] 108 23882 6795 75 0 0 0 trivial-rewrite
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285731] [25127] 108 25127 6818 81 0 0 0 cleanup
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285734] [25128] 108 25128 11107 158 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285737] [25138] 108 25138 11107 159 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285740] [25149] 108 25149 11107 158 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285744] [26567] 108 26567 11107 157 1 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285747] [26582] 108 26582 6792 73 1 0 0 scache
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285750] [26583] 108 26583 11108 157 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285753] [26676] 108 26676 6792 76 1 0 0 pickup
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285756] [27725] 1012 27725 800979 737880 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285759] [27769] 108 27769 11108 159 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285762] [27773] 108 27773 11108 158 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285765] [27777] 108 27777 11076 137 0 0 0 smtp
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285768] [27996] 1012 27996 68580 5606 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285771] [28028] 1012 28028 1099 24 0 0 0 sh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285774] [28029] 1012 28029 68825 5870 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285777] [28030] 1012 28030 1099 24 0 0 0 sh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285780] [28031] 1012 28031 68825 5873 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285783] [28042] 1012 28042 1099 26 0 0 0 sh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285786] [28043] 1012 28043 1099 25 0 0 0 sh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285789] [28044] 1012 28044 14040 63 0 0 0 git
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285792] [28045] 1012 28045 4736 66 0 0 0 git
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285795] [28046] 1012 28046 68499 5552 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285798] [28047] 1012 28047 68499 5551 1 0 0 php
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285801] [28050] 1012 28050 1099 25 0 0 0 sh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285804] [28051] 1012 28051 10901 165 1 0 0 ssh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285808] [28052] 1012 28052 1099 25 1 0 0 sh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285810] [28053] 1012 28053 10901 164 1 0 0 ssh
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285813] Out of memory: Kill process 27725 (php) score 770 or sacrifice child
Jul 15 23:56:21 ip-10-61-174-46 kernel: [1459477.285824] Killed process 27725 (php) total-vm:3203916kB, anon-rss:2951520kB, file-rss:0kB
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596205] ssh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596209] ssh cpuset=/ mems_allowed=0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596212] Pid: 1930, comm: ssh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596213] Call Trace:
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596220] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596223] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596225] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596228] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596233] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596236] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596239] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596242] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596245] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596247] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596250] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596253] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596255] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596258] [<ffffffff811425e0>] ? find_vma+0x30/0x70
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596261] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596263] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596265] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596268] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596270] Mem-Info:
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596271] Node 0 DMA per-cpu:
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596273] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596274] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596275] Node 0 DMA32 per-cpu:
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596277] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596278] CPU 1: hi: 186, btch: 31 usd: 12
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596281] active_anon:932478 inactive_anon:1393 isolated_anon:0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596282] active_file:0 inactive_file:0 isolated_file:0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596282] unevictable:0 dirty:3 writeback:0 unstable:0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596283] free:4044 slab_reclaimable:2628 slab_unreclaimable:2671
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596284] mapped:1343 shmem:1405 pagetables:3393 bounce:0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596285] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596292] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596295] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3729912kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:12kB writeback:0kB mapped:5372kB shmem:5620kB slab_reclaimable:10512kB slab_unreclaimable:10684kB kernel_stack:776kB pagetables:13572kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12 all_unreclaimable? yes
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596301] lowmem_reserve[]: 0 0 0 0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596304] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596310] Node 0 DMA32: 991*4kB 215*8kB 6*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596317] 1449 total pagecache pages
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596318] 0 pages in swap cache
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596320] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596321] Free swap = 0kB
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.596322] Total swap = 0kB
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601473] 985072 pages RAM
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601476] 24998 pages reserved
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601477] 3804 pages shared
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601478] 953279 pages non-shared
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601480] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601489] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601492] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601496] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601499] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601502] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601505] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601508] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601512] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601516] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601519] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601522] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601525] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601528] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601531] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601535] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601538] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601541] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601544] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601547] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601551] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601554] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601557] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601560] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601563] [13468] 0 13468 89584 2496 1 0 0 php5-fpm
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601566] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601569] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601572] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601576] [26676] 108 26676 6792 76 0 0 0 pickup
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601579] [28148] 1012 28148 814228 751157 0 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601583] [ 764] 108 764 6818 74 1 0 0 cleanup
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601586] [ 766] 108 766 6795 71 1 0 0 trivial-rewrite
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601589] [ 767] 108 767 11107 158 0 0 0 smtp
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601592] [ 771] 108 771 11107 159 1 0 0 smtp
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601595] [ 1793] 1012 1793 68580 5606 1 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601598] [ 1918] 1012 1918 1099 25 0 0 0 sh
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601601] [ 1919] 1012 1919 68825 5872 0 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601604] [ 1925] 1012 1925 1099 26 0 0 0 sh
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601607] [ 1926] 1012 1926 4658 58 0 0 0 git
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601610] [ 1927] 1012 1927 68499 5552 0 0 0 php
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601613] [ 1929] 1012 1929 1099 25 0 0 0 sh
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601616] [ 1930] 1012 1930 9829 137 0 0 0 ssh
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601619] Out of memory: Kill process 28148 (php) score 784 or sacrifice child
Jul 16 00:05:12 ip-10-61-174-46 kernel: [1460007.601630] Killed process 28148 (php) total-vm:3256912kB, anon-rss:3004628kB, file-rss:0kB
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152141] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152146] ntpd cpuset=/ mems_allowed=0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152149] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152151] Call Trace:
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152159] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152162] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152166] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152169] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152173] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152178] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152180] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152184] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152192] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152195] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152198] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152202] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152206] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152209] Mem-Info:
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152210] Node 0 DMA per-cpu:
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152216] Node 0 DMA32 per-cpu:
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152220] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152223] active_anon:932982 inactive_anon:1393 isolated_anon:0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152224] active_file:13 inactive_file:20 isolated_file:0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152225] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152226] free:4048 slab_reclaimable:2527 slab_unreclaimable:2632
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152227] mapped:1395 shmem:1405 pagetables:3075 bounce:0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152229] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152236] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152239] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731928kB inactive_anon:5572kB active_file:52kB inactive_file:80kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5580kB shmem:5620kB slab_reclaimable:10108kB slab_unreclaimable:10528kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12894 all_unreclaimable? yes
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152245] lowmem_reserve[]: 0 0 0 0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152248] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152254] Node 0 DMA32: 914*4kB 185*8kB 46*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152261] 1487 total pagecache pages
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152262] 0 pages in swap cache
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152264] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152265] Free swap = 0kB
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.152266] Total swap = 0kB
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157337] 985072 pages RAM
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157340] 24998 pages reserved
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157341] 3810 pages shared
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157342] 953289 pages non-shared
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157344] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157353] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157356] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157360] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157362] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157366] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157369] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157372] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157376] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157380] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157383] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157386] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157389] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157393] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157396] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157399] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157402] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157405] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157408] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157411] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157415] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157417] [ 657] 1012 657 69242 6267 1 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157421] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157424] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157427] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157430] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157433] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157437] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157440] [26676] 108 26676 6792 76 0 0 0 pickup
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157444] [ 2072] 1012 2072 827031 763800 1 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157447] [ 3594] 1012 3594 68580 5604 1 0 0 php
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157449] Out of memory: Kill process 2072 (php) score 797 or sacrifice child
Jul 16 00:07:29 ip-10-61-174-46 kernel: [1460145.157460] Killed process 2072 (php) total-vm:3308124kB, anon-rss:3055200kB, file-rss:0kB
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788555] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788559] php cpuset=/ mems_allowed=0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788562] Pid: 3739, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788563] Call Trace:
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788570] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788573] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788575] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788578] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788581] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788585] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788587] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788590] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788592] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788596] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788598] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788601] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788603] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788606] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788609] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788611] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788613] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788617] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788618] Mem-Info:
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788619] Node 0 DMA per-cpu:
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788621] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788623] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788624] Node 0 DMA32 per-cpu:
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788625] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788627] CPU 1: hi: 186, btch: 31 usd: 75
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788629] active_anon:933311 inactive_anon:1393 isolated_anon:0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788630] active_file:0 inactive_file:0 isolated_file:0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788631] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788631] free:4019 slab_reclaimable:2226 slab_unreclaimable:2622
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788632] mapped:1366 shmem:1405 pagetables:3063 bounce:0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788633] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788640] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788643] Node 0 DMA32 free:7740kB min:7848kB low:9808kB high:11772kB active_anon:3733244kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:8904kB slab_unreclaimable:10488kB kernel_stack:728kB pagetables:12252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:79 all_unreclaimable? yes
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788650] lowmem_reserve[]: 0 0 0 0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788652] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788659] Node 0 DMA32: 687*4kB 171*8kB 68*16kB 16*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7764kB
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788665] 1426 total pagecache pages
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788667] 0 pages in swap cache
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788668] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788669] Free swap = 0kB
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.788670] Total swap = 0kB
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794286] 985072 pages RAM
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794288] 24998 pages reserved
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794289] 3784 pages shared
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794291] 953271 pages non-shared
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794293] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794302] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794306] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794309] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794312] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794315] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794318] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794322] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794325] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794329] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794332] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794335] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794339] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794342] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794345] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794348] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794351] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794354] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794357] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794360] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794363] [ 654] 1012 654 70351 5765 0 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794366] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794369] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794372] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794376] [13468] 0 13468 89584 2498 0 0 0 php5-fpm
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794379] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794382] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794385] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794388] [26676] 108 26676 6792 76 0 0 0 pickup
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794392] [ 3739] 1012 3739 827362 764153 0 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794395] [ 4331] 1012 4331 68580 5604 1 0 0 php
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794398] Out of memory: Kill process 3739 (php) score 797 or sacrifice child
Jul 16 00:08:41 ip-10-61-174-46 kernel: [1460216.794410] Killed process 3739 (php) total-vm:3309448kB, anon-rss:3056612kB, file-rss:0kB
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652476] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652480] php cpuset=/ mems_allowed=0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652484] Pid: 9809, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652486] Call Trace:
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652494] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652497] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652500] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652504] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652510] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652513] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652517] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652520] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652524] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652527] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652531] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652534] [<ffffffff8100634e>] ? pte_mfn_to_pfn+0x7e/0x110
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652537] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652540] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652543] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652547] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652550] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652553] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652557] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652559] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652562] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652566] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652569] Mem-Info:
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652570] Node 0 DMA per-cpu:
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652572] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652574] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652575] Node 0 DMA32 per-cpu:
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652576] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652578] CPU 1: hi: 186, btch: 31 usd: 90
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652581] active_anon:932359 inactive_anon:1393 isolated_anon:0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652582] active_file:23 inactive_file:12 isolated_file:0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652582] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652583] free:4018 slab_reclaimable:2861 slab_unreclaimable:2666
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652584] mapped:1366 shmem:1405 pagetables:3266 bounce:0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652587] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652595] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652598] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3729436kB inactive_anon:5572kB active_file:92kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:11444kB slab_unreclaimable:10664kB kernel_stack:776kB pagetables:13064kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:280 all_unreclaimable? yes
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652605] lowmem_reserve[]: 0 0 0 0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652607] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652614] Node 0 DMA32: 996*4kB 183*8kB 11*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652620] 1464 total pagecache pages
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652622] 0 pages in swap cache
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652623] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652625] Free swap = 0kB
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.652626] Total swap = 0kB
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657783] 985072 pages RAM
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657785] 24998 pages reserved
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657786] 3788 pages shared
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657787] 953283 pages non-shared
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657789] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657798] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657801] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657805] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657808] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657811] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657814] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657817] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657821] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657825] [ 790] 101 790 63477 641 0 0 0 rsyslogd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657828] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657831] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657834] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657837] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657841] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657844] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657847] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657850] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657853] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657856] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657859] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657862] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657865] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657868] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657871] [13468] 0 13468 89584 2529 0 0 0 php5-fpm
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657875] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657877] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657881] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657884] [26676] 108 26676 6792 76 1 0 0 pickup
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657887] [ 5426] 1012 5426 805747 742502 1 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657891] [ 9111] 108 9111 6818 74 0 0 0 cleanup
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657894] [ 9112] 108 9112 6795 70 0 0 0 trivial-rewrite
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657897] [ 9114] 108 9114 11107 158 1 0 0 smtp
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657900] [ 9122] 108 9122 11107 158 0 0 0 smtp
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657903] [ 9136] 108 9136 11107 158 0 0 0 smtp
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657906] [ 9671] 1012 9671 77831 14876 0 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657909] [ 9809] 1012 9809 73753 10808 1 0 0 php
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657911] Out of memory: Kill process 5426 (php) score 774 or sacrifice child
Jul 16 00:15:04 ip-10-61-174-46 kernel: [1460599.657922] Killed process 5426 (php) total-vm:3222988kB, anon-rss:2970008kB, file-rss:0kB
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928147] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928151] php5-fpm cpuset=/ mems_allowed=0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928154] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928156] Call Trace:
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928162] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928165] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928167] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928170] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928175] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928178] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928181] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928183] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928186] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928190] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928192] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928195] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928200] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928202] [<ffffffff8113f9d1>] ? handle_mm_fault+0x1/0x370
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928206] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928209] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928213] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928216] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928218] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928221] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928224] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928227] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928228] Mem-Info:
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928230] Node 0 DMA per-cpu:
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928231] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928233] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928234] Node 0 DMA32 per-cpu:
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928236] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928237] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928239] active_anon:933135 inactive_anon:1393 isolated_anon:0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928240] active_file:0 inactive_file:0 isolated_file:0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928241] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928241] free:4042 slab_reclaimable:2234 slab_unreclaimable:2681
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928242] mapped:1359 shmem:1405 pagetables:3185 bounce:0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928244] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928250] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928253] Node 0 DMA32 free:7832kB min:7848kB low:9808kB high:11772kB active_anon:3732540kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:8936kB slab_unreclaimable:10724kB kernel_stack:768kB pagetables:12740kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928260] lowmem_reserve[]: 0 0 0 0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928262] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928269] Node 0 DMA32: 704*4kB 211*8kB 43*16kB 17*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7912kB
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928275] 1464 total pagecache pages
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928277] 0 pages in swap cache
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928278] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928279] Free swap = 0kB
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.928280] Total swap = 0kB
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933721] 985072 pages RAM
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933723] 24998 pages reserved
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933724] 3800 pages shared
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933725] 953318 pages non-shared
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933727] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933736] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933740] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933743] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933746] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933749] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933752] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933756] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933759] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933763] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933766] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933769] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933773] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933776] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933779] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933782] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933785] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933788] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933792] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933795] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933798] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933801] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933804] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933807] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933810] [13468] 0 13468 89584 2537 0 0 0 php5-fpm
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933813] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933816] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933819] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933823] [26676] 108 26676 6792 76 1 0 0 pickup
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933826] [ 9111] 108 9111 6818 74 0 0 0 cleanup
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933829] [ 9112] 108 9112 6795 70 1 0 0 trivial-rewrite
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933832] [ 9114] 108 9114 11107 158 1 0 0 smtp
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933835] [ 9122] 108 9122 11107 158 0 0 0 smtp
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933838] [ 9136] 108 9136 11107 158 0 0 0 smtp
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933843] [10005] 1012 10005 826594 763326 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933846] [10984] 1012 10984 68580 5605 1 0 0 php
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933849] Out of memory: Kill process 10005 (php) score 796 or sacrifice child
Jul 16 00:16:30 ip-10-61-174-46 kernel: [1460685.933859] Killed process 10005 (php) total-vm:3306376kB, anon-rss:3053304kB, file-rss:0kB
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052132] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052136] php cpuset=/ mems_allowed=0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052139] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052141] Call Trace:
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052147] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052150] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052152] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052155] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052158] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052161] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052164] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052167] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052170] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052173] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052175] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052178] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052180] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052183] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052186] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052190] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052192] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052195] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052198] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052199] Mem-Info:
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052201] Node 0 DMA per-cpu:
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052203] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052204] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052205] Node 0 DMA32 per-cpu:
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052207] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052208] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052210] active_anon:932782 inactive_anon:1393 isolated_anon:0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052211] active_file:0 inactive_file:13 isolated_file:0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052212] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052212] free:4047 slab_reclaimable:2616 slab_unreclaimable:2624
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052213] mapped:1353 shmem:1405 pagetables:3214 bounce:0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052215] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052221] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052224] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731128kB inactive_anon:5572kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10464kB slab_unreclaimable:10496kB kernel_stack:768kB pagetables:12856kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:19221 all_unreclaimable? yes
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052231] lowmem_reserve[]: 0 0 0 0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052233] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052240] Node 0 DMA32: 1366*4kB 55*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052246] 1439 total pagecache pages
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052248] 0 pages in swap cache
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052249] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052250] Free swap = 0kB
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.052251] Total swap = 0kB
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057621] 985072 pages RAM
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057623] 24998 pages reserved
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057625] 3791 pages shared
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057626] 953301 pages non-shared
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057628] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057636] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057640] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057643] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057646] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057649] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057652] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057656] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057659] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057663] [ 790] 101 790 63477 641 1 0 0 rsyslogd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057666] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057669] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057673] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057676] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057679] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057682] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057685] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057688] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057691] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057695] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057698] [ 654] 1012 654 70351 5765 1 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057701] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057704] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057707] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057710] [13468] 0 13468 89584 2522 1 0 0 php5-fpm
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057713] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057716] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057720] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057723] [26676] 108 26676 6792 76 1 0 0 pickup
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057726] [12750] 1012 12750 826022 762811 0 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057729] [14216] 108 14216 6818 74 1 0 0 cleanup
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057732] [14217] 108 14217 6795 70 1 0 0 trivial-rewrite
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057735] [14218] 108 14218 11107 159 0 0 0 smtp
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057738] [14222] 108 14222 11107 158 0 0 0 smtp
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057741] [14229] 108 14229 11107 158 0 0 0 smtp
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057744] [15278] 108 15278 11107 159 1 0 0 smtp
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057747] [15910] 1012 15910 68580 5604 0 0 0 php
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057750] Out of memory: Kill process 12750 (php) score 796 or sacrifice child
Jul 16 00:22:02 ip-10-61-174-46 kernel: [1461018.057762] Killed process 12750 (php) total-vm:3304088kB, anon-rss:3051244kB, file-rss:0kB
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884328] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884332] php cpuset=/ mems_allowed=0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884335] Pid: 16616, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884336] Call Trace:
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884343] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884346] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884348] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884351] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884356] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884358] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884362] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884364] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884367] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884370] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884372] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884375] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884377] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884379] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884382] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884384] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884387] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884390] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884394] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884396] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884399] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884402] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884403] Mem-Info:
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884405] Node 0 DMA per-cpu:
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884406] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884408] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884409] Node 0 DMA32 per-cpu:
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884411] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884412] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884415] active_anon:932938 inactive_anon:1393 isolated_anon:0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884415] active_file:0 inactive_file:6 isolated_file:0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884416] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884417] free:4039 slab_reclaimable:2469 slab_unreclaimable:2606
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884417] mapped:1359 shmem:1405 pagetables:3187 bounce:0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884419] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884425] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884428] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3731752kB inactive_anon:5572kB active_file:0kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:9876kB slab_unreclaimable:10424kB kernel_stack:768kB pagetables:12748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:162 all_unreclaimable? yes
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884435] lowmem_reserve[]: 0 0 0 0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884437] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884444] Node 0 DMA32: 723*4kB 180*8kB 44*16kB 19*32kB 2*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884451] 1437 total pagecache pages
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884452] 0 pages in swap cache
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884453] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884454] Free swap = 0kB
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.884455] Total swap = 0kB
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889639] 985072 pages RAM
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889642] 24998 pages reserved
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889643] 3791 pages shared
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889644] 953286 pages non-shared
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889646] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889655] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889659] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889662] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889665] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889668] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889671] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889675] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889678] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889682] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889685] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889688] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889691] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889694] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889697] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889700] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889703] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889707] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889710] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889713] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889716] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889719] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889722] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889725] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889728] [13468] 0 13468 89584 2507 1 0 0 php5-fpm
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889731] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889734] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889738] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889741] [26676] 108 26676 6792 76 0 0 0 pickup
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889744] [16147] 1012 16147 820130 756861 0 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889747] [16616] 1012 16616 74820 11870 0 0 0 php
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889750] [16845] 108 16845 6818 73 0 0 0 cleanup
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889753] [16846] 108 16846 6795 71 0 0 0 trivial-rewrite
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889756] [16847] 108 16847 11107 159 1 0 0 smtp
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889759] [16851] 108 16851 11107 158 1 0 0 smtp
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889762] [16855] 108 16855 11107 158 1 0 0 smtp
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889765] Out of memory: Kill process 16147 (php) score 789 or sacrifice child
Jul 16 00:22:57 ip-10-61-174-46 kernel: [1461072.889777] Killed process 16147 (php) total-vm:3280520kB, anon-rss:3027444kB, file-rss:0kB
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268144] php cpuset=/ mems_allowed=0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268147] Pid: 16936, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268149] Call Trace:
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268167] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268173] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268176] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268183] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268187] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268190] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268194] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268197] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268200] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268203] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268206] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268211] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268213] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268216] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268221] Mem-Info:
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268223] Node 0 DMA per-cpu:
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268224] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268226] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268227] Node 0 DMA32 per-cpu:
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268229] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268230] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268232] active_anon:932711 inactive_anon:1393 isolated_anon:0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268233] active_file:0 inactive_file:0 isolated_file:0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268234] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268234] free:4044 slab_reclaimable:2607 slab_unreclaimable:2633
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268235] mapped:1364 shmem:1405 pagetables:3272 bounce:0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268237] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268243] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268246] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3730844kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:10428kB slab_unreclaimable:10532kB kernel_stack:784kB pagetables:13088kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268253] lowmem_reserve[]: 0 0 0 0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268255] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268262] Node 0 DMA32: 993*4kB 219*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7900kB
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268269] 1478 total pagecache pages
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268270] 0 pages in swap cache
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268271] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268272] Free swap = 0kB
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.268273] Total swap = 0kB
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273339] 985072 pages RAM
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273341] 24998 pages reserved
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273342] 3782 pages shared
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273343] 953337 pages non-shared
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273345] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273354] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273358] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273361] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273364] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273367] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273370] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273373] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273377] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273381] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273384] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273387] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273390] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273394] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273397] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273400] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273403] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273406] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273409] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273412] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273415] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273419] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273422] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273425] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273428] [13468] 0 13468 89584 2540 1 0 0 php5-fpm
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273431] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273434] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273438] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273441] [16936] 1012 16936 821948 758626 0 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273444] [18774] 108 18774 6792 69 1 0 0 pickup
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273447] [18778] 108 18778 6818 74 1 0 0 cleanup
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273450] [18779] 108 18779 6795 71 1 0 0 trivial-rewrite
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273454] [18780] 108 18780 11107 158 0 0 0 smtp
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273457] [18784] 108 18784 11107 157 0 0 0 smtp
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273460] [18788] 108 18788 11107 158 1 0 0 smtp
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273463] [19349] 1012 19349 68580 5605 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273466] [19455] 1012 19455 1099 25 1 0 0 sh
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273469] [19456] 1012 19456 67198 4288 1 0 0 php
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273471] Out of memory: Kill process 16936 (php) score 791 or sacrifice child
Jul 16 00:25:59 ip-10-61-174-46 kernel: [1461255.273482] Killed process 16936 (php) total-vm:3287792kB, anon-rss:3034500kB, file-rss:4kB
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812131] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812135] php cpuset=/ mems_allowed=0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812138] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812140] Call Trace:
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812152] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812155] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812158] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812162] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812167] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812171] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812181] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812184] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812191] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812193] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812196] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812199] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812201] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812203] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812207] [<ffffffff8106bc3c>] ? do_wait+0x12c/0x260
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812210] [<ffffffff8106cf4b>] ? sys_wait4+0xab/0xf0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812212] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812214] Mem-Info:
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812215] Node 0 DMA per-cpu:
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812217] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812218] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812219] Node 0 DMA32 per-cpu:
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812221] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812222] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812225] active_anon:932995 inactive_anon:1393 isolated_anon:0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812226] active_file:18 inactive_file:16 isolated_file:0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812226] unevictable:0 dirty:0 writeback:14 unstable:0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812227] free:4048 slab_reclaimable:2450 slab_unreclaimable:2653
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812228] mapped:1361 shmem:1405 pagetables:3168 bounce:0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812229] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812236] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812239] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731980kB inactive_anon:5572kB active_file:72kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:56kB mapped:5444kB shmem:5620kB slab_reclaimable:9800kB slab_unreclaimable:10612kB kernel_stack:768kB pagetables:12672kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10972 all_unreclaimable? yes
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812245] lowmem_reserve[]: 0 0 0 0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812248] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812254] Node 0 DMA32: 913*4kB 273*8kB 5*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7964kB
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812261] 1452 total pagecache pages
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812262] 0 pages in swap cache
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812264] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812265] Free swap = 0kB
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.812266] Total swap = 0kB
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817447] 985072 pages RAM
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817450] 24998 pages reserved
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817451] 3775 pages shared
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817452] 953330 pages non-shared
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817454] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817463] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817466] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817469] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817472] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817476] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817479] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817482] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817486] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817490] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817493] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817496] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817499] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817502] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817505] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817508] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817511] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817514] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817517] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817521] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817524] [ 654] 1012 654 70351 5766 0 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817527] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817530] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817533] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817536] [13468] 0 13468 89584 2525 1 0 0 php5-fpm
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817539] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817542] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817545] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817549] [18774] 108 18774 6792 69 1 0 0 pickup
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817552] [18778] 108 18778 6818 74 1 0 0 cleanup
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817555] [18779] 108 18779 6795 71 1 0 0 trivial-rewrite
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817558] [18780] 108 18780 11107 158 0 0 0 smtp
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817561] [18784] 108 18784 11107 157 0 0 0 smtp
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817564] [18788] 108 18788 11107 158 1 0 0 smtp
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817569] [19498] 1012 19498 826429 763166 0 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817573] [19734] 1012 19734 68580 5604 1 0 0 php
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817575] Out of memory: Kill process 19498 (php) score 796 or sacrifice child
Jul 16 00:26:35 ip-10-61-174-46 kernel: [1461290.817587] Killed process 19498 (php) total-vm:3305716kB, anon-rss:3052664kB, file-rss:0kB
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820106] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820109] php cpuset=/ mems_allowed=0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820112] Pid: 20108, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820114] Call Trace:
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820121] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820123] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820125] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820128] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820132] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820135] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820137] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820141] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820143] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820146] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820149] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820151] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820154] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820157] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820159] Mem-Info:
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820160] Node 0 DMA per-cpu:
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820162] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820163] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820164] Node 0 DMA32 per-cpu:
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820166] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820167] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820170] active_anon:933016 inactive_anon:1393 isolated_anon:0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820170] active_file:11 inactive_file:0 isolated_file:0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820171] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820172] free:4048 slab_reclaimable:2389 slab_unreclaimable:2619
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820172] mapped:1376 shmem:1405 pagetables:3215 bounce:0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820174] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820180] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820183] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732064kB inactive_anon:5572kB active_file:44kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5504kB shmem:5620kB slab_reclaimable:9556kB slab_unreclaimable:10476kB kernel_stack:776kB pagetables:12860kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:648 all_unreclaimable? yes
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820190] lowmem_reserve[]: 0 0 0 0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820192] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820199] Node 0 DMA32: 856*4kB 224*8kB 35*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7920kB
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820206] 1451 total pagecache pages
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820207] 0 pages in swap cache
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820208] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820209] Free swap = 0kB
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.820210] Total swap = 0kB
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825660] 985072 pages RAM
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825662] 24998 pages reserved
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825663] 3786 pages shared
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825665] 953329 pages non-shared
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825667] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825676] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825679] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825682] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825686] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825689] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825692] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825695] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825699] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825703] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825706] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825709] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825712] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825715] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825718] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825722] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825725] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825728] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825731] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825734] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825737] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825740] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825744] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825747] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825750] [13468] 0 13468 89584 2510 1 0 0 php5-fpm
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825753] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825756] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825759] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825763] [18774] 108 18774 6792 75 1 0 0 pickup
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825765] [20108] 1012 20108 826252 763038 0 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825769] [20890] 108 20890 6818 74 1 0 0 cleanup
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825772] [20891] 108 20891 6795 71 0 0 0 trivial-rewrite
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825775] [20892] 108 20892 11107 158 0 0 0 smtp
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825778] [20904] 108 20904 11107 159 0 0 0 smtp
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825781] [20908] 108 20908 11107 158 1 0 0 smtp
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825784] [20914] 108 20914 11107 159 1 0 0 smtp
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825787] [21337] 1012 21337 68580 5606 1 0 0 php
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825790] Out of memory: Kill process 20108 (php) score 796 or sacrifice child
Jul 16 00:28:25 ip-10-61-174-46 kernel: [1461400.825800] Killed process 20108 (php) total-vm:3305008kB, anon-rss:3052116kB, file-rss:36kB
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952095] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952099] php cpuset=/ mems_allowed=0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952102] Pid: 21623, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952103] Call Trace:
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952110] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952113] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952115] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952118] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952121] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952124] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952126] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952129] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952131] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952135] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952137] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952140] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952143] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952146] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952148] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952151] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952153] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952155] Mem-Info:
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952156] Node 0 DMA per-cpu:
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952158] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952159] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952160] Node 0 DMA32 per-cpu:
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952162] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952164] CPU 1: hi: 186, btch: 31 usd: 4
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952166] active_anon:933071 inactive_anon:1393 isolated_anon:0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952167] active_file:16 inactive_file:18 isolated_file:0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952167] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952168] free:4048 slab_reclaimable:2446 slab_unreclaimable:2582
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952169] mapped:1397 shmem:1405 pagetables:3139 bounce:0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952170] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952177] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952180] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732284kB inactive_anon:5572kB active_file:64kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5588kB shmem:5620kB slab_reclaimable:9784kB slab_unreclaimable:10328kB kernel_stack:752kB pagetables:12556kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:7603 all_unreclaimable? yes
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952186] lowmem_reserve[]: 0 0 0 0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952189] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952195] Node 0 DMA32: 794*4kB 253*8kB 38*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7952kB
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952202] 1460 total pagecache pages
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952203] 0 pages in swap cache
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952205] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952206] Free swap = 0kB
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.952206] Total swap = 0kB
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957354] 985072 pages RAM
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957356] 24998 pages reserved
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957357] 3810 pages shared
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957358] 953296 pages non-shared
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957360] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957369] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957373] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957376] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957379] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957382] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957386] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957389] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957392] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957396] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957399] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957402] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957406] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957409] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957412] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957415] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957418] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957421] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957424] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957428] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957431] [ 654] 1012 654 70351 5767 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957434] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957437] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957440] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957443] [13468] 0 13468 89584 2495 0 0 0 php5-fpm
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957446] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957449] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957453] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957456] [18774] 108 18774 6792 75 1 0 0 pickup
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957460] [21623] 1012 21623 826775 763595 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957463] [22731] 108 22731 6818 73 1 0 0 cleanup
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957466] [22732] 108 22732 6795 71 1 0 0 trivial-rewrite
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957469] [22733] 108 22733 11107 158 0 0 0 smtp
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957472] [22764] 1012 22764 68580 5605 0 0 0 php
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957475] Out of memory: Kill process 21623 (php) score 797 or sacrifice child
Jul 16 00:29:56 ip-10-61-174-46 kernel: [1461491.957485] Killed process 21623 (php) total-vm:3307100kB, anon-rss:3054300kB, file-rss:80kB
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587560] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587565] php cpuset=/ mems_allowed=0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587568] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587570] Call Trace:
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587579] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587582] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587585] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587589] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587593] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587598] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587601] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587605] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587608] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587612] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587615] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587618] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587622] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587627] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587631] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587635] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587639] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587642] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587645] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587649] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587652] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587655] Mem-Info:
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587656] Node 0 DMA per-cpu:
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587659] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587660] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587661] Node 0 DMA32 per-cpu:
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587663] CPU 0: hi: 186, btch: 31 usd: 46
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587664] CPU 1: hi: 186, btch: 31 usd: 110
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587667] active_anon:932333 inactive_anon:1393 isolated_anon:0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587668] active_file:0 inactive_file:18 isolated_file:23
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587669] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587669] free:4039 slab_reclaimable:2943 slab_unreclaimable:2658
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587670] mapped:1352 shmem:1405 pagetables:3184 bounce:0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587672] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587678] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587681] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3729332kB inactive_anon:5572kB active_file:0kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):92kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:11772kB slab_unreclaimable:10632kB kernel_stack:768kB pagetables:12736kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:210 all_unreclaimable? yes
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587688] lowmem_reserve[]: 0 0 0 0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587690] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587697] Node 0 DMA32: 1435*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7788kB
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587704] 1442 total pagecache pages
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587705] 0 pages in swap cache
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587706] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587707] Free swap = 0kB
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.587708] Total swap = 0kB
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592866] 985072 pages RAM
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592869] 24998 pages reserved
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592870] 3774 pages shared
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592871] 953219 pages non-shared
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592873] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592882] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592886] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592889] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592892] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592895] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592899] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592902] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592905] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592909] [ 790] 101 790 63477 642 0 0 0 rsyslogd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592912] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592915] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592919] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592922] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592925] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592928] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592931] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592934] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592937] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592941] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592944] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592947] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592950] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592953] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592956] [13468] 0 13468 89584 2517 0 0 0 php5-fpm
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592959] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592962] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592965] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592969] [ 3583] 1012 3583 819710 756537 0 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592972] [ 5667] 108 5667 6818 73 0 0 0 cleanup
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592975] [ 5668] 108 5668 6795 70 0 0 0 trivial-rewrite
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592979] [ 5669] 108 5669 11107 158 0 0 0 smtp
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592981] [ 5673] 108 5673 11107 159 0 0 0 smtp
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592984] [ 5677] 108 5677 11107 158 1 0 0 smtp
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592988] [ 6726] 1012 6726 74497 11610 1 0 0 php
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592991] [ 6947] 108 6947 6792 70 1 0 0 pickup
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.592994] Out of memory: Kill process 3583 (php) score 789 or sacrifice child
Jul 16 00:48:31 ip-10-61-174-46 kernel: [1462606.593005] Killed process 3583 (php) total-vm:3278840kB, anon-rss:3026148kB, file-rss:0kB
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132151] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132155] php cpuset=/ mems_allowed=0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132158] Pid: 6726, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132159] Call Trace:
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132166] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132171] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132174] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132177] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132186] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132189] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132191] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132196] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132201] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132203] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132206] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132212] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132214] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132216] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132221] Mem-Info:
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132222] Node 0 DMA per-cpu:
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132224] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132225] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132226] Node 0 DMA32 per-cpu:
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132228] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132229] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132232] active_anon:932442 inactive_anon:1393 isolated_anon:0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132232] active_file:23 inactive_file:24 isolated_file:0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132233] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132234] free:4048 slab_reclaimable:2377 slab_unreclaimable:2669
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132235] mapped:1370 shmem:1405 pagetables:3567 bounce:0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132236] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132243] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132246] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729768kB inactive_anon:5572kB active_file:92kB inactive_file:96kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:9508kB slab_unreclaimable:10676kB kernel_stack:824kB pagetables:14268kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9246 all_unreclaimable? yes
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132252] lowmem_reserve[]: 0 0 0 0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132255] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132262] Node 0 DMA32: 672*4kB 70*8kB 87*16kB 23*32kB 5*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132268] 1462 total pagecache pages
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132270] 0 pages in swap cache
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132271] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132272] Free swap = 0kB
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.132273] Total swap = 0kB
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137462] 985072 pages RAM
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137465] 24998 pages reserved
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137466] 3809 pages shared
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137467] 953312 pages non-shared
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137469] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137478] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137481] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137484] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137488] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137491] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137494] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137497] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137501] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137505] [ 790] 101 790 63477 642 0 0 0 rsyslogd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137508] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137511] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137514] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137517] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137520] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137523] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137527] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137530] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137533] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137536] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137539] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137543] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137546] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137549] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137552] [13468] 0 13468 89584 2502 1 0 0 php5-fpm
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137555] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137558] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137561] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137565] [ 5667] 108 5667 6818 73 0 0 0 cleanup
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137568] [ 5668] 108 5668 6795 74 0 0 0 trivial-rewrite
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137571] [ 5669] 108 5669 11107 158 0 0 0 smtp
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137574] [ 5673] 108 5673 11107 159 0 0 0 smtp
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137577] [ 5677] 108 5677 11107 158 1 0 0 smtp
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137580] [ 6726] 1012 6726 617709 554511 0 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137583] [ 6947] 108 6947 6792 70 1 0 0 pickup
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137586] [ 7091] 1012 7091 264521 201273 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137589] [ 7231] 1012 7231 68580 5604 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137592] [ 7253] 1012 7253 1099 25 0 0 0 sh
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137595] [ 7254] 1012 7254 65054 2603 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137598] [ 7255] 1012 7255 1099 24 0 0 0 sh
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137601] [ 7256] 1012 7256 1099 24 1 0 0 sh
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137604] [ 7257] 1012 7257 65053 2609 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137607] [ 7258] 1012 7258 64213 1581 1 0 0 php
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137609] Out of memory: Kill process 6726 (php) score 578 or sacrifice child
Jul 16 00:48:54 ip-10-61-174-46 kernel: [1462630.137620] Killed process 6726 (php) total-vm:2470836kB, anon-rss:2217956kB, file-rss:88kB
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572127] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572132] php cpuset=/ mems_allowed=0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572134] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572136] Call Trace:
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572143] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572145] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572147] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572150] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572155] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572158] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572162] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572164] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572167] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572171] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572174] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572176] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572179] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572182] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572184] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572187] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572189] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572191] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572194] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572195] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572198] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572200] Mem-Info:
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572201] Node 0 DMA per-cpu:
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572203] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572204] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572205] Node 0 DMA32 per-cpu:
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572207] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572208] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572211] active_anon:932839 inactive_anon:1393 isolated_anon:0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572211] active_file:0 inactive_file:21 isolated_file:0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572212] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572213] free:4047 slab_reclaimable:2382 slab_unreclaimable:2649
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572213] mapped:1365 shmem:1405 pagetables:3280 bounce:0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572215] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572222] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572225] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731356kB inactive_anon:5572kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:9528kB slab_unreclaimable:10596kB kernel_stack:776kB pagetables:13120kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9353 all_unreclaimable? yes
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572231] lowmem_reserve[]: 0 0 0 0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572234] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572240] Node 0 DMA32: 739*4kB 149*8kB 91*16kB 9*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572247] 1426 total pagecache pages
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572248] 0 pages in swap cache
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572250] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572251] Free swap = 0kB
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.572252] Total swap = 0kB
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577316] 985072 pages RAM
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577318] 24998 pages reserved
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577319] 3786 pages shared
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577320] 953308 pages non-shared
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577323] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577332] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577335] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577339] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577342] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577345] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577348] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577351] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577355] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577358] [ 790] 101 790 63477 642 0 0 0 rsyslogd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577362] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577365] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577368] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577371] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577374] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577377] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577380] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577383] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577386] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577390] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577393] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577396] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577399] [ 658] 1012 658 200573 135991 1 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577402] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577405] [13468] 0 13468 89584 2535 0 0 0 php5-fpm
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577408] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577411] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577414] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577418] [ 5667] 108 5667 6818 73 0 0 0 cleanup
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577421] [ 5668] 108 5668 6795 74 1 0 0 trivial-rewrite
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577424] [ 5669] 108 5669 11107 158 0 0 0 smtp
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577427] [ 5673] 108 5673 11107 159 0 0 0 smtp
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577430] [ 5677] 108 5677 11107 158 1 0 0 smtp
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577433] [ 6947] 108 6947 6792 70 1 0 0 pickup
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577436] [ 7091] 1012 7091 820642 757423 0 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577439] [ 7231] 1012 7231 68580 5604 1 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577442] [ 7407] 1012 7407 68580 5604 0 0 0 php
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577445] Out of memory: Kill process 7091 (php) score 790 or sacrifice child
Jul 16 00:49:09 ip-10-61-174-46 kernel: [1462644.577457] Killed process 7091 (php) total-vm:3282568kB, anon-rss:3029692kB, file-rss:0kB
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796142] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796146] php cpuset=/ mems_allowed=0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796149] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796151] Call Trace:
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796158] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796168] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796172] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796177] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796180] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796183] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796185] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796193] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796196] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796200] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796203] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796205] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796208] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796210] Mem-Info:
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796211] Node 0 DMA per-cpu:
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796214] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796215] Node 0 DMA32 per-cpu:
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796217] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796218] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796221] active_anon:932758 inactive_anon:1393 isolated_anon:0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796221] active_file:0 inactive_file:0 isolated_file:0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796222] unevictable:0 dirty:11 writeback:0 unstable:0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796223] free:4047 slab_reclaimable:2770 slab_unreclaimable:2669
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796223] mapped:1350 shmem:1405 pagetables:3078 bounce:0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796225] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796232] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796235] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731032kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:44kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:11080kB slab_unreclaimable:10676kB kernel_stack:728kB pagetables:12312kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796241] lowmem_reserve[]: 0 0 0 0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796244] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796251] Node 0 DMA32: 1267*4kB 92*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796257] 1461 total pagecache pages
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796259] 0 pages in swap cache
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796260] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796261] Free swap = 0kB
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.796262] Total swap = 0kB
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801542] 985072 pages RAM
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801544] 24998 pages reserved
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801545] 3783 pages shared
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801547] 953321 pages non-shared
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801549] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801556] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801560] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801563] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801566] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801569] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801573] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801576] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801580] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801583] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801587] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801590] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801593] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801596] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801599] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801602] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801605] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801609] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801612] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801615] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801618] [ 654] 1012 654 70351 5767 0 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801621] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801624] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801627] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801630] [13468] 0 13468 89584 2520 1 0 0 php5-fpm
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801633] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801636] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801639] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801643] [ 6947] 108 6947 6792 76 1 0 0 pickup
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801646] [ 7557] 1012 7557 826865 763583 1 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801649] [15174] 1012 15174 68580 5604 1 0 0 php
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801651] Out of memory: Kill process 7557 (php) score 796 or sacrifice child
Jul 16 00:57:30 ip-10-61-174-46 kernel: [1463145.801662] Killed process 7557 (php) total-vm:3307460kB, anon-rss:3054328kB, file-rss:4kB
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812126] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812130] php cpuset=/ mems_allowed=0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812133] Pid: 15512, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812135] Call Trace:
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812141] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812144] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812146] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812149] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812152] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812155] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812158] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812161] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812164] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812167] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812169] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812171] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812174] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812176] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812180] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812182] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812184] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812187] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812189] Mem-Info:
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812190] Node 0 DMA per-cpu:
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812192] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812194] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812195] Node 0 DMA32 per-cpu:
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812196] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812198] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812200] active_anon:932611 inactive_anon:1393 isolated_anon:0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812201] active_file:28 inactive_file:17 isolated_file:0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812201] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812202] free:4048 slab_reclaimable:2829 slab_unreclaimable:2671
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812203] mapped:1373 shmem:1405 pagetables:3174 bounce:0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812204] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812211] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812214] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730444kB inactive_anon:5572kB active_file:112kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5492kB shmem:5620kB slab_reclaimable:11316kB slab_unreclaimable:10684kB kernel_stack:744kB pagetables:12696kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2331 all_unreclaimable? yes
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812220] lowmem_reserve[]: 0 0 0 0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812223] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812230] Node 0 DMA32: 1276*4kB 78*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812236] 1456 total pagecache pages
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812237] 0 pages in swap cache
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812239] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812240] Free swap = 0kB
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.812241] Total swap = 0kB
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817383] 985072 pages RAM
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817385] 24998 pages reserved
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817386] 3813 pages shared
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817387] 953318 pages non-shared
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817389] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817398] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817401] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817405] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817408] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817411] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817414] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817417] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817421] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817425] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817428] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817431] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817434] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817437] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817440] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817444] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817447] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817450] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817453] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817456] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817459] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817462] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817466] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817469] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817472] [13468] 0 13468 89584 2505 1 0 0 php5-fpm
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817475] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817478] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817481] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817485] [ 6947] 108 6947 6792 76 0 0 0 pickup
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817488] [15512] 1012 15512 822684 759343 0 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817491] [19131] 1012 19131 68580 5606 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817494] [19398] 1012 19398 1099 24 1 0 0 sh
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817497] [19399] 1012 19399 66484 4033 1 0 0 php
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817500] Out of memory: Kill process 15512 (php) score 792 or sacrifice child
Jul 16 01:01:28 ip-10-61-174-46 kernel: [1463383.817511] Killed process 15512 (php) total-vm:3290736kB, anon-rss:3037244kB, file-rss:128kB
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408130] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408134] php cpuset=/ mems_allowed=0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408137] Pid: 19475, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408139] Call Trace:
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408145] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408148] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408153] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408156] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408160] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408162] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408165] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408167] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408171] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408173] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408175] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408178] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408180] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408184] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408186] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408188] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408191] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408193] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408195] Mem-Info:
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408197] Node 0 DMA per-cpu:
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408198] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408200] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408201] Node 0 DMA32 per-cpu:
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408202] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408204] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408206] active_anon:932177 inactive_anon:1393 isolated_anon:0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408207] active_file:0 inactive_file:0 isolated_file:0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408208] unevictable:0 dirty:6 writeback:0 unstable:0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408208] free:4048 slab_reclaimable:2327 slab_unreclaimable:2729
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408209] mapped:1357 shmem:1405 pagetables:3819 bounce:0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408210] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408217] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408220] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728708kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:24kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9308kB slab_unreclaimable:10916kB kernel_stack:848kB pagetables:15276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408226] lowmem_reserve[]: 0 0 0 0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408229] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408236] Node 0 DMA32: 688*4kB 216*8kB 41*16kB 11*32kB 3*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408242] 1494 total pagecache pages
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408244] 0 pages in swap cache
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408245] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408246] Free swap = 0kB
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.408247] Total swap = 0kB
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413396] 985072 pages RAM
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413399] 24998 pages reserved
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413400] 3807 pages shared
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413401] 953315 pages non-shared
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413403] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413412] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413416] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413419] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413422] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413425] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413428] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413432] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413435] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413440] [ 790] 101 790 63477 642 1 0 0 rsyslogd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413443] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413446] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413449] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413452] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413455] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413459] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413462] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413465] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413468] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413471] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413474] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413477] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413480] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413483] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413487] [13468] 0 13468 89584 2490 1 0 0 php5-fpm
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413490] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413493] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413496] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413500] [ 6947] 108 6947 6792 76 0 0 0 pickup
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413503] [19475] 1012 19475 791072 727801 0 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413506] [20895] 1012 20895 68580 5606 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413509] [21206] 1012 21206 1099 25 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413512] [21207] 1012 21207 68825 5872 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413515] [21208] 1012 21208 1099 24 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413518] [21209] 1012 21209 68825 5875 0 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413521] [21210] 1012 21210 1099 24 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413524] [21211] 1012 21211 68825 5874 0 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413527] [21223] 1012 21223 1099 25 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413530] [21225] 1012 21225 4651 57 1 0 0 git
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413533] [21226] 1012 21226 68499 5552 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413536] [21230] 1012 21230 1099 25 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413539] [21231] 1012 21231 4661 54 1 0 0 git
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413542] [21232] 1012 21232 1099 26 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413545] [21233] 1012 21233 68499 5552 1 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413548] [21234] 1012 21234 4650 52 1 0 0 git
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413551] [21235] 1012 21235 68499 5552 0 0 0 php
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413554] [21239] 1012 21239 1099 24 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413557] [21240] 1012 21240 10901 165 0 0 0 ssh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413560] [21241] 1012 21241 1099 24 1 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413563] [21242] 1012 21242 10901 175 1 0 0 ssh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413566] [21243] 1012 21243 1099 25 0 0 0 sh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413569] [21244] 1012 21244 10901 175 1 0 0 ssh
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413572] Out of memory: Kill process 19475 (php) score 759 or sacrifice child
Jul 16 01:03:02 ip-10-61-174-46 kernel: [1463478.413583] Killed process 19475 (php) total-vm:3164288kB, anon-rss:2911204kB, file-rss:0kB
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117201] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117205] ntpd cpuset=/ mems_allowed=0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117208] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117209] Call Trace:
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117217] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117219] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117221] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117224] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117227] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117231] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117233] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117236] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117239] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117242] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117244] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117247] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117250] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117253] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117257] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117258] Mem-Info:
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117259] Node 0 DMA per-cpu:
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117261] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117263] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117264] Node 0 DMA32 per-cpu:
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117265] CPU 0: hi: 186, btch: 31 usd: 47
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117267] CPU 1: hi: 186, btch: 31 usd: 70
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117269] active_anon:931408 inactive_anon:1393 isolated_anon:0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117270] active_file:25 inactive_file:48 isolated_file:0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117271] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117271] free:4033 slab_reclaimable:2827 slab_unreclaimable:2765
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117272] mapped:1363 shmem:1405 pagetables:3887 bounce:0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117273] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117280] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117283] Node 0 DMA32 free:7796kB min:7848kB low:9808kB high:11772kB active_anon:3725632kB inactive_anon:5572kB active_file:100kB inactive_file:192kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:11308kB slab_unreclaimable:11060kB kernel_stack:928kB pagetables:15548kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:463 all_unreclaimable? yes
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117289] lowmem_reserve[]: 0 0 0 0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117292] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117299] Node 0 DMA32: 1113*4kB 160*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117305] 1487 total pagecache pages
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117307] 0 pages in swap cache
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117308] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117309] Free swap = 0kB
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.117310] Total swap = 0kB
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122502] 985072 pages RAM
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122505] 24998 pages reserved
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122506] 3817 pages shared
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122507] 953193 pages non-shared
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122509] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122517] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122521] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122524] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122527] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122530] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122533] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122537] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122540] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122544] [ 790] 101 790 63477 643 0 0 0 rsyslogd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122547] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122550] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122553] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122557] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122560] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122563] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122566] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122569] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122572] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122575] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122578] [ 654] 1012 654 70351 5766 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122581] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122584] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122587] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122591] [13468] 0 13468 89584 2511 0 0 0 php5-fpm
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122594] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122596] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122600] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122604] [ 6947] 108 6947 6792 76 1 0 0 pickup
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122607] [24749] 1012 24749 784411 721124 1 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122610] [ 2345] 108 2345 6818 81 0 0 0 cleanup
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122613] [ 2348] 108 2348 6795 75 0 0 0 trivial-rewrite
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122616] [ 3198] 108 3198 11107 159 0 0 0 smtp
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122619] [ 5820] 1012 5820 74134 11168 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122622] [ 5962] 108 5962 11107 159 0 0 0 smtp
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122625] [ 5966] 1012 5966 1099 25 0 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122628] [ 5967] 1012 5967 68825 5875 1 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122631] [ 5968] 1012 5968 1099 25 0 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122634] [ 5969] 1012 5969 68825 5869 1 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122637] [ 5970] 1012 5970 1099 25 0 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122640] [ 5971] 1012 5971 68825 5871 0 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122643] [ 5983] 1012 5983 1099 25 1 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122646] [ 5984] 1012 5984 4676 58 1 0 0 git
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122649] [ 5986] 1012 5986 68499 5551 1 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122652] [ 5990] 1012 5990 1099 25 1 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122655] [ 5991] 1012 5991 1099 26 1 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122658] [ 5992] 1012 5992 4738 65 1 0 0 git
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122661] [ 5993] 1012 5993 4776 58 1 0 0 git
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122664] [ 5994] 1012 5994 68499 5552 1 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122667] [ 5995] 1012 5995 68499 5551 1 0 0 php
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122670] [ 5999] 1012 5999 1099 25 1 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122673] [ 6000] 1012 6000 1099 25 1 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122676] [ 6001] 1012 6001 1099 25 1 0 0 sh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122679] [ 6002] 1012 6002 10348 133 0 0 0 ssh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122682] [ 6003] 1012 6003 10348 127 1 0 0 ssh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122685] [ 6004] 1012 6004 10348 120 1 0 0 ssh
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122687] Out of memory: Kill process 24749 (php) score 752 or sacrifice child
Jul 16 01:21:30 ip-10-61-174-46 kernel: [1464586.122698] Killed process 24749 (php) total-vm:3137644kB, anon-rss:2884496kB, file-rss:0kB
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892316] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892320] php cpuset=/ mems_allowed=0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892323] Pid: 6108, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892325] Call Trace:
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892332] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892334] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892337] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892339] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892343] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892346] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892348] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892351] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892356] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892358] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892361] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892364] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892366] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892369] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892372] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892374] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892376] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892379] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892381] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892383] Mem-Info:
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892384] Node 0 DMA per-cpu:
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892386] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892387] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892389] Node 0 DMA32 per-cpu:
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892390] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892392] CPU 1: hi: 186, btch: 31 usd: 15
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892394] active_anon:932642 inactive_anon:1393 isolated_anon:0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892395] active_file:0 inactive_file:0 isolated_file:0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892395] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892396] free:4043 slab_reclaimable:2865 slab_unreclaimable:2671
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892397] mapped:1380 shmem:1405 pagetables:3073 bounce:0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892398] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892405] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892408] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3730568kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5520kB shmem:5620kB slab_reclaimable:11460kB slab_unreclaimable:10684kB kernel_stack:728kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24 all_unreclaimable? yes
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892414] lowmem_reserve[]: 0 0 0 0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892417] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892423] Node 0 DMA32: 1469*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892430] 1450 total pagecache pages
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892431] 0 pages in swap cache
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892432] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892433] Free swap = 0kB
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.892434] Total swap = 0kB
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897719] 985072 pages RAM
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897722] 24998 pages reserved
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897723] 3799 pages shared
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897724] 953276 pages non-shared
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897726] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897735] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897738] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897741] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897744] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897748] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897751] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897754] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897758] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897762] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897765] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897768] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897771] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897774] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897777] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897780] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897783] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897786] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897790] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897793] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897796] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897799] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897802] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897805] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897808] [13468] 0 13468 89584 2531 0 0 0 php5-fpm
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897811] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897814] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897817] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897821] [ 6108] 1012 6108 826709 763467 0 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897824] [ 8537] 108 8537 6792 70 0 0 0 pickup
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897827] [ 8758] 1012 8758 68580 5605 0 0 0 php
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897830] Out of memory: Kill process 6108 (php) score 796 or sacrifice child
Jul 16 01:24:40 ip-10-61-174-46 kernel: [1464775.897840] Killed process 6108 (php) total-vm:3306836kB, anon-rss:3053868kB, file-rss:0kB
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411500] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411504] php5-fpm cpuset=/ mems_allowed=0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411507] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411509] Call Trace:
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411516] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411519] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411521] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411524] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411527] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411530] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411532] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411535] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411539] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411541] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411545] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411547] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411549] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411553] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411556] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411560] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411564] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411567] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411569] Mem-Info:
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411570] Node 0 DMA per-cpu:
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411572] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411574] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411575] Node 0 DMA32 per-cpu:
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411576] CPU 0: hi: 186, btch: 31 usd: 84
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411578] CPU 1: hi: 186, btch: 31 usd: 13
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411580] active_anon:932717 inactive_anon:1393 isolated_anon:0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411581] active_file:17 inactive_file:12 isolated_file:14
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411582] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411582] free:4036 slab_reclaimable:2738 slab_unreclaimable:2662
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411583] mapped:1370 shmem:1405 pagetables:3069 bounce:0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411585] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411591] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411594] Node 0 DMA32 free:7808kB min:7848kB low:9808kB high:11772kB active_anon:3730868kB inactive_anon:5572kB active_file:68kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):56kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:10952kB slab_unreclaimable:10648kB kernel_stack:728kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:220 all_unreclaimable? yes
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411601] lowmem_reserve[]: 0 0 0 0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411603] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411610] Node 0 DMA32: 1323*4kB 55*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411617] 1415 total pagecache pages
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411618] 0 pages in swap cache
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411619] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411620] Free swap = 0kB
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.411621] Total swap = 0kB
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416770] 985072 pages RAM
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416772] 24998 pages reserved
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416774] 3788 pages shared
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416775] 953263 pages non-shared
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416777] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416786] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416789] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416792] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416795] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416799] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416802] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416805] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416808] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416812] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416815] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416818] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416822] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416825] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416828] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416831] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416834] [ 1051] 0 1051 201629 848 1 0 0 collectd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416838] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416841] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416844] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416847] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416850] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416853] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416856] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416859] [13468] 0 13468 89584 2498 0 0 0 php5-fpm
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416862] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416865] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416868] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416872] [ 8537] 108 8537 6792 76 0 0 0 pickup
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416875] [16485] 1012 16485 826786 763527 1 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416878] [17357] 1012 17357 68580 5605 0 0 0 php
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416881] Out of memory: Kill process 16485 (php) score 796 or sacrifice child
Jul 16 01:33:44 ip-10-61-174-46 kernel: [1465320.416891] Killed process 16485 (php) total-vm:3307144kB, anon-rss:3054108kB, file-rss:0kB
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.115994] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.115999] ntpd cpuset=/ mems_allowed=0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116002] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116004] Call Trace:
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116011] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116014] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116016] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116019] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116022] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116025] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116028] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116031] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116033] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116036] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116039] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116041] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116045] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116048] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116053] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116065] Mem-Info:
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116067] Node 0 DMA per-cpu:
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116069] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116071] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116073] Node 0 DMA32 per-cpu:
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116075] CPU 0: hi: 186, btch: 31 usd: 110
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116077] CPU 1: hi: 186, btch: 31 usd: 10
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116081] active_anon:932785 inactive_anon:1393 isolated_anon:0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116082] active_file:18 inactive_file:14 isolated_file:24
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116083] unevictable:0 dirty:24 writeback:0 unstable:0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116083] free:4042 slab_reclaimable:2453 slab_unreclaimable:2683
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116084] mapped:1375 shmem:1405 pagetables:3248 bounce:0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116086] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116095] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116099] Node 0 DMA32 free:7832kB min:7848kB low:9808kB high:11772kB active_anon:3731140kB inactive_anon:5572kB active_file:72kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):96kB present:3862656kB mlocked:0kB dirty:96kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:9812kB slab_unreclaimable:10732kB kernel_stack:760kB pagetables:12992kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:261 all_unreclaimable? yes
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116108] lowmem_reserve[]: 0 0 0 0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116112] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116122] Node 0 DMA32: 904*4kB 186*8kB 25*16kB 9*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116132] 1426 total pagecache pages
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116134] 0 pages in swap cache
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116135] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116137] Free swap = 0kB
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.116138] Total swap = 0kB
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121452] 985072 pages RAM
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121454] 24998 pages reserved
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121455] 3784 pages shared
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121456] 953228 pages non-shared
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121458] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121465] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121468] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121470] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121472] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121475] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121477] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121479] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121482] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121485] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121487] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121489] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121492] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121494] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121496] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121498] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121501] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121503] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121505] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121507] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121510] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121512] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121514] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121516] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121518] [13468] 0 13468 89584 2520 1 0 0 php5-fpm
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121520] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121522] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121525] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121528] [ 8537] 108 8537 6792 76 0 0 0 pickup
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121530] [17766] 1012 17766 815394 752122 0 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121532] [18822] 1012 18822 68580 5605 1 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121534] [18904] 1012 18904 1099 24 0 0 0 sh
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121536] [18905] 1012 18905 68662 5714 1 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121538] [18906] 1012 18906 1099 24 1 0 0 sh
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121540] [18907] 1012 18907 68662 5714 1 0 0 php
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121543] Out of memory: Kill process 17766 (php) score 785 or sacrifice child
Jul 16 01:35:16 ip-10-61-174-46 kernel: [1465412.121552] Killed process 17766 (php) total-vm:3261576kB, anon-rss:3008484kB, file-rss:4kB
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200124] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200128] php cpuset=/ mems_allowed=0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200131] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200132] Call Trace:
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200139] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200142] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200150] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200154] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200156] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200159] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200162] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200164] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200171] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200176] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200179] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200182] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200185] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200188] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200190] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200193] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200196] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200197] Mem-Info:
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200198] Node 0 DMA per-cpu:
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200200] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200202] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200203] Node 0 DMA32 per-cpu:
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200204] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200206] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200208] active_anon:933005 inactive_anon:1393 isolated_anon:0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200209] active_file:0 inactive_file:0 isolated_file:0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200209] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200210] free:4048 slab_reclaimable:2561 slab_unreclaimable:2658
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200211] mapped:1375 shmem:1405 pagetables:3052 bounce:0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200212] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200219] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200222] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732020kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:10244kB slab_unreclaimable:10632kB kernel_stack:728kB pagetables:12208kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200228] lowmem_reserve[]: 0 0 0 0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200230] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200237] Node 0 DMA32: 1234*4kB 104*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7864kB
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200244] 1449 total pagecache pages
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200245] 0 pages in swap cache
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200246] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200247] Free swap = 0kB
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.200248] Total swap = 0kB
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205364] 985072 pages RAM
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205366] 24998 pages reserved
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205367] 3797 pages shared
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205369] 953339 pages non-shared
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205370] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205379] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205383] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205386] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205389] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205393] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205396] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205399] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205403] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205407] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205410] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205413] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205416] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205420] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205423] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205426] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205429] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205432] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205435] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205438] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205441] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205444] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205447] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205450] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205454] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205457] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205460] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205463] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205467] [ 8537] 108 8537 6792 76 1 0 0 pickup
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205470] [18982] 1012 18982 827066 763825 0 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205473] [26191] 1012 26191 68580 5606 1 0 0 php
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205476] Out of memory: Kill process 18982 (php) score 797 or sacrifice child
Jul 16 01:43:11 ip-10-61-174-46 kernel: [1465887.205487] Killed process 18982 (php) total-vm:3308264kB, anon-rss:3055300kB, file-rss:0kB
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124151] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124155] php cpuset=/ mems_allowed=0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124158] Pid: 28472, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124159] Call Trace:
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124166] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124168] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124171] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124173] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124178] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124181] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124184] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124186] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124189] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124192] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124195] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124197] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124200] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124202] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124205] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124207] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124210] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124212] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124215] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124216] Mem-Info:
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124218] Node 0 DMA per-cpu:
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124220] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124221] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124222] Node 0 DMA32 per-cpu:
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124224] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124225] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124227] active_anon:932501 inactive_anon:1393 isolated_anon:0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124228] active_file:22 inactive_file:34 isolated_file:0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124229] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124229] free:4048 slab_reclaimable:2848 slab_unreclaimable:2722
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124230] mapped:1383 shmem:1405 pagetables:3170 bounce:0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124232] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124238] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124241] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730004kB inactive_anon:5572kB active_file:88kB inactive_file:136kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5532kB shmem:5620kB slab_reclaimable:11392kB slab_unreclaimable:10888kB kernel_stack:768kB pagetables:12680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15946 all_unreclaimable? yes
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124248] lowmem_reserve[]: 0 0 0 0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124250] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124257] Node 0 DMA32: 1419*4kB 21*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8004kB
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124263] 1477 total pagecache pages
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124265] 0 pages in swap cache
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124266] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124267] Free swap = 0kB
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.124268] Total swap = 0kB
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129655] 985072 pages RAM
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129657] 24998 pages reserved
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129658] 3798 pages shared
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129659] 953297 pages non-shared
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129661] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129670] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129674] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129677] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129680] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129683] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129686] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129689] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129693] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129697] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129700] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129703] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129706] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129710] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129713] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129716] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129719] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129722] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129725] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129728] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129731] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129734] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129737] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129740] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129744] [13468] 0 13468 89584 2524 1 0 0 php5-fpm
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129747] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129750] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129753] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129756] [ 8537] 108 8537 6792 76 1 0 0 pickup
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129760] [27925] 108 27925 6818 73 0 0 0 cleanup
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129763] [27926] 108 27926 6795 71 1 0 0 trivial-rewrite
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129766] [27927] 108 27927 11107 158 1 0 0 smtp
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129769] [27938] 108 27938 11107 158 0 0 0 smtp
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129772] [28472] 1012 28472 825877 762701 0 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129775] [28723] 108 28723 11107 158 1 0 0 smtp
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129778] [28951] 1012 28951 68580 5604 1 0 0 php
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129781] Out of memory: Kill process 28472 (php) score 796 or sacrifice child
Jul 16 01:46:13 ip-10-61-174-46 kernel: [1466069.129792] Killed process 28472 (php) total-vm:3303508kB, anon-rss:3050720kB, file-rss:84kB
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876318] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876323] php cpuset=/ mems_allowed=0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876325] Pid: 29172, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876327] Call Trace:
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876334] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876336] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876339] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876341] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876346] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876349] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876353] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876355] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876358] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876360] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876363] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876366] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876368] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876371] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876373] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876377] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876379] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876381] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876384] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876387] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876388] Mem-Info:
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876389] Node 0 DMA per-cpu:
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876391] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876392] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876394] Node 0 DMA32 per-cpu:
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876395] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876396] CPU 1: hi: 186, btch: 31 usd: 17
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876399] active_anon:932253 inactive_anon:1393 isolated_anon:0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876400] active_file:0 inactive_file:0 isolated_file:0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876400] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876401] free:4039 slab_reclaimable:2921 slab_unreclaimable:2681
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876402] mapped:1384 shmem:1405 pagetables:3317 bounce:0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876403] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876410] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876413] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3729012kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5536kB shmem:5620kB slab_reclaimable:11684kB slab_unreclaimable:10724kB kernel_stack:784kB pagetables:13268kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:18 all_unreclaimable? yes
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876419] lowmem_reserve[]: 0 0 0 0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876422] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876428] Node 0 DMA32: 1381*4kB 23*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876435] 1448 total pagecache pages
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876436] 0 pages in swap cache
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876438] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876439] Free swap = 0kB
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.876440] Total swap = 0kB
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881891] 985072 pages RAM
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881893] 24998 pages reserved
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881895] 3789 pages shared
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881896] 953305 pages non-shared
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881898] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881906] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881910] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881913] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881916] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881919] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881922] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881925] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881929] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881933] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881936] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881939] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881943] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881946] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881949] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881952] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881955] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881958] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881961] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881965] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881968] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881971] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881974] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881977] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881980] [13468] 0 13468 89584 2500 1 0 0 php5-fpm
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881983] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881986] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881990] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881993] [ 8537] 108 8537 6792 76 1 0 0 pickup
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881996] [29172] 1012 29172 814595 751348 0 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.881999] [ 2378] 1012 2378 68580 5604 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882002] [ 2615] 1012 2615 1099 25 1 0 0 sh
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882005] [ 2616] 1012 2616 68825 5871 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882009] [ 2622] 1012 2622 1099 26 1 0 0 sh
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882011] [ 2623] 1012 2623 4776 58 1 0 0 git
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882014] [ 2624] 1012 2624 68499 5552 1 0 0 php
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882017] [ 2626] 1012 2626 1099 25 1 0 0 sh
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882020] [ 2627] 1012 2627 10901 175 0 0 0 ssh
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882023] Out of memory: Kill process 29172 (php) score 784 or sacrifice child
Jul 16 01:52:35 ip-10-61-174-46 kernel: [1466450.882035] Killed process 29172 (php) total-vm:3258380kB, anon-rss:3005392kB, file-rss:0kB
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388282] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388286] php cpuset=/ mems_allowed=0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388289] Pid: 2737, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388290] Call Trace:
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388297] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388299] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388301] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388304] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388307] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388311] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388313] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388316] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388318] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388322] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388324] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388327] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388329] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388331] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388335] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388337] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388339] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388342] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388344] Mem-Info:
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388345] Node 0 DMA per-cpu:
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388347] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388348] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388349] Node 0 DMA32 per-cpu:
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388351] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388352] CPU 1: hi: 186, btch: 31 usd: 27
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388355] active_anon:933388 inactive_anon:1393 isolated_anon:0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388356] active_file:3 inactive_file:0 isolated_file:0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388356] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388357] free:4024 slab_reclaimable:2207 slab_unreclaimable:2616
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388358] mapped:1369 shmem:1405 pagetables:3058 bounce:0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388359] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388366] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388369] Node 0 DMA32 free:7760kB min:7848kB low:9808kB high:11772kB active_anon:3733552kB inactive_anon:5572kB active_file:12kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5620kB slab_reclaimable:8828kB slab_unreclaimable:10464kB kernel_stack:728kB pagetables:12232kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:95 all_unreclaimable? yes
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388375] lowmem_reserve[]: 0 0 0 0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388378] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388384] Node 0 DMA32: 642*4kB 145*8kB 106*16kB 7*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7760kB
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388391] 1409 total pagecache pages
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388392] 0 pages in swap cache
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388393] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388394] Free swap = 0kB
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.388395] Total swap = 0kB
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393804] 985072 pages RAM
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393806] 24998 pages reserved
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393808] 3782 pages shared
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393809] 953322 pages non-shared
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393811] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393820] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393823] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393827] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393830] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393833] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393836] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393839] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393843] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393847] [ 790] 101 790 63477 643 1 0 0 rsyslogd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393850] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393853] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393857] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393860] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393863] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393866] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393869] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393873] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393876] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393879] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393882] [ 654] 1012 654 70351 5766 1 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393885] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393888] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393891] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393894] [13468] 0 13468 89584 2485 1 0 0 php5-fpm
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393897] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393901] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393904] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393907] [ 8537] 108 8537 6792 76 0 0 0 pickup
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393911] [ 2737] 1012 2737 827490 764225 0 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393914] [ 3329] 1012 3329 68580 5604 1 0 0 php
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393916] Out of memory: Kill process 2737 (php) score 797 or sacrifice child
Jul 16 01:53:40 ip-10-61-174-46 kernel: [1466516.393928] Killed process 2737 (php) total-vm:3309960kB, anon-rss:3056896kB, file-rss:4kB
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528365] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528369] php cpuset=/ mems_allowed=0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528371] Pid: 4482, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528373] Call Trace:
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528380] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528382] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528385] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528387] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528391] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528394] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528396] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528399] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528402] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528405] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528407] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528410] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528413] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528415] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528418] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528421] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528422] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528426] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528428] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528430] Mem-Info:
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528431] Node 0 DMA per-cpu:
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528433] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528434] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528435] Node 0 DMA32 per-cpu:
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528437] CPU 0: hi: 186, btch: 31 usd: 31
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528438] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528441] active_anon:932853 inactive_anon:1393 isolated_anon:0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528442] active_file:0 inactive_file:0 isolated_file:0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528442] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528443] free:4040 slab_reclaimable:2443 slab_unreclaimable:2687
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528444] mapped:1363 shmem:1405 pagetables:3267 bounce:0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528445] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528452] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528455] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3731412kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5620kB slab_reclaimable:9772kB slab_unreclaimable:10748kB kernel_stack:784kB pagetables:13068kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:130 all_unreclaimable? yes
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528461] lowmem_reserve[]: 0 0 0 0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528463] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528470] Node 0 DMA32: 1258*4kB 93*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528477] 1449 total pagecache pages
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528478] 0 pages in swap cache
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528479] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528480] Free swap = 0kB
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.528481] Total swap = 0kB
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533843] 985072 pages RAM
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533846] 24998 pages reserved
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533847] 3810 pages shared
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533848] 953267 pages non-shared
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533850] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533858] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533862] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533865] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533868] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533871] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533875] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533878] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533881] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533885] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533888] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533892] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533895] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533898] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533901] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533904] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533907] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533911] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533914] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533917] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533920] [ 654] 1012 654 70351 5779 1 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533923] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533926] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533929] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533932] [13468] 0 13468 89584 2526 1 0 0 php5-fpm
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533935] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533938] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533942] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533945] [ 8537] 108 8537 6792 76 1 0 0 pickup
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533948] [ 3628] 1012 3628 77701 14836 0 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533951] [ 3630] 1012 3630 816161 752926 0 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533954] [ 3686] 108 3686 6818 72 1 0 0 cleanup
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533958] [ 3689] 108 3689 6795 71 0 0 0 trivial-rewrite
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533961] [ 3690] 108 3690 11107 159 0 0 0 smtp
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533963] [ 3707] 108 3707 11107 158 0 0 0 smtp
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533966] [ 3711] 108 3711 11107 158 1 0 0 smtp
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533970] [ 4481] 1012 4481 1099 25 1 0 0 sh
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533972] [ 4482] 1012 4482 44042 869 0 0 0 php
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533975] Out of memory: Kill process 3630 (php) score 785 or sacrifice child
Jul 16 01:54:40 ip-10-61-174-46 kernel: [1466575.533986] Killed process 3630 (php) total-vm:3264644kB, anon-rss:3011704kB, file-rss:0kB
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068124] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068128] php cpuset=/ mems_allowed=0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068131] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068132] Call Trace:
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068139] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068141] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068144] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068146] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068152] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068154] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068158] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068160] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068163] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068166] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068169] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068171] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068174] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068176] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068179] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068182] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068184] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068186] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068190] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068191] Mem-Info:
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068193] Node 0 DMA per-cpu:
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068194] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068196] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068197] Node 0 DMA32 per-cpu:
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068198] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068200] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068202] active_anon:932723 inactive_anon:1393 isolated_anon:0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068203] active_file:0 inactive_file:0 isolated_file:0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068204] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068204] free:4048 slab_reclaimable:2796 slab_unreclaimable:2663
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068205] mapped:1365 shmem:1405 pagetables:3077 bounce:0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068207] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068213] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068216] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730892kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:11184kB slab_unreclaimable:10652kB kernel_stack:728kB pagetables:12308kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068222] lowmem_reserve[]: 0 0 0 0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068225] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068232] Node 0 DMA32: 1434*4kB 7*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068238] 1472 total pagecache pages
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068240] 0 pages in swap cache
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068241] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068242] Free swap = 0kB
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.068243] Total swap = 0kB
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073743] 985072 pages RAM
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073745] 24998 pages reserved
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073746] 3788 pages shared
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073748] 953349 pages non-shared
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073750] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073758] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073762] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073765] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073768] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073771] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073774] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073778] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073781] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073785] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073788] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073791] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073794] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073797] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073800] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073804] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073807] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073810] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073813] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073816] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073819] [ 654] 1012 654 70351 5767 1 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073822] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073825] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073828] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073831] [13468] 0 13468 89584 2498 1 0 0 php5-fpm
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073834] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073837] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073841] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073844] [ 3628] 1012 3628 826805 763535 1 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073847] [ 9739] 108 9739 6792 70 1 0 0 pickup
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073850] [15560] 1012 15560 68580 5606 1 0 0 php
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073853] Out of memory: Kill process 3628 (php) score 796 or sacrifice child
Jul 16 02:06:49 ip-10-61-174-46 kernel: [1467305.073864] Killed process 3628 (php) total-vm:3307220kB, anon-rss:3054140kB, file-rss:0kB
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360118] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360122] php cpuset=/ mems_allowed=0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360125] Pid: 15963, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360127] Call Trace:
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360133] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360136] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360138] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360141] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360146] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360149] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360154] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360157] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360160] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360163] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360166] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360168] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360171] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360174] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360176] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360179] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360182] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360183] Mem-Info:
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360184] Node 0 DMA per-cpu:
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360186] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360188] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360189] Node 0 DMA32 per-cpu:
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360191] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360192] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360194] active_anon:933070 inactive_anon:1393 isolated_anon:0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360195] active_file:0 inactive_file:0 isolated_file:0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360196] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360196] free:4048 slab_reclaimable:2271 slab_unreclaimable:2687
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360197] mapped:1352 shmem:1405 pagetables:3221 bounce:0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360199] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360205] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360208] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732280kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9084kB slab_unreclaimable:10748kB kernel_stack:776kB pagetables:12884kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360214] lowmem_reserve[]: 0 0 0 0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360217] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360224] Node 0 DMA32: 754*4kB 293*8kB 23*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360230] 1500 total pagecache pages
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360231] 0 pages in swap cache
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360233] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360234] Free swap = 0kB
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.360235] Total swap = 0kB
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365651] 985072 pages RAM
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365654] 24998 pages reserved
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365655] 3811 pages shared
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365656] 953301 pages non-shared
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365658] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365666] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365670] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365673] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365676] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365680] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365683] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365686] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365689] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365693] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365696] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365700] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365703] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365706] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365709] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365712] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365716] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365719] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365722] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365725] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365728] [ 654] 1012 654 70351 5768 1 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365731] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365734] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365737] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365740] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365743] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365746] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365749] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365753] [ 9739] 108 9739 6792 76 0 0 0 pickup
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365756] [15963] 1012 15963 826360 763111 0 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365759] [15985] 108 15985 6818 73 0 0 0 cleanup
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365762] [15986] 108 15986 6795 71 0 0 0 trivial-rewrite
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365766] [15987] 108 15987 11107 158 1 0 0 smtp
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365769] [15999] 108 15999 11107 158 1 0 0 smtp
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365772] [16001] 108 16001 11107 158 1 0 0 smtp
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365775] [16012] 108 16012 11107 158 0 0 0 smtp
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365778] [16350] 1012 16350 68580 5605 1 0 0 php
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365781] Out of memory: Kill process 15963 (php) score 796 or sacrifice child
Jul 16 02:07:30 ip-10-61-174-46 kernel: [1467346.365792] Killed process 15963 (php) total-vm:3305440kB, anon-rss:3052444kB, file-rss:0kB
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976565] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976569] php cpuset=/ mems_allowed=0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976571] Pid: 20228, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976573] Call Trace:
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976580] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976582] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976584] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976587] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976590] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976594] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976596] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976599] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976601] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976605] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976607] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976610] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976612] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976615] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976618] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976620] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976622] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976625] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976628] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976629] Mem-Info:
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976631] Node 0 DMA per-cpu:
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976632] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976634] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976635] Node 0 DMA32 per-cpu:
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976637] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976638] CPU 1: hi: 186, btch: 31 usd: 54
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976640] active_anon:932369 inactive_anon:1393 isolated_anon:0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976641] active_file:0 inactive_file:0 isolated_file:0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976642] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976642] free:4018 slab_reclaimable:2870 slab_unreclaimable:2678
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976643] mapped:1362 shmem:1405 pagetables:3259 bounce:0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976645] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976651] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976654] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3729476kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:11480kB slab_unreclaimable:10712kB kernel_stack:792kB pagetables:13036kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:63 all_unreclaimable? yes
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976661] lowmem_reserve[]: 0 0 0 0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976663] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976670] Node 0 DMA32: 1278*4kB 72*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976676] 1432 total pagecache pages
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976678] 0 pages in swap cache
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976679] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976680] Free swap = 0kB
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.976681] Total swap = 0kB
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982116] 985072 pages RAM
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982118] 24998 pages reserved
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982120] 3789 pages shared
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982121] 953286 pages non-shared
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982123] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982132] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982135] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982138] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982142] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982145] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982148] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982151] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982155] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982159] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982162] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982165] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982169] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982172] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982175] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982178] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982182] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982185] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982188] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982191] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982195] [ 654] 1012 654 70351 5768 1 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982198] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982201] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982204] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982207] [13468] 0 13468 89584 2516 1 0 0 php5-fpm
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982210] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982213] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982216] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982220] [ 9739] 108 9739 6792 76 0 0 0 pickup
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982223] [15986] 108 15986 6795 75 1 0 0 trivial-rewrite
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982227] [18742] 108 18742 6818 74 1 0 0 cleanup
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982230] [18743] 108 18743 11107 160 1 0 0 smtp
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982233] [18747] 108 18747 11107 159 1 0 0 smtp
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982236] [18751] 108 18751 11107 158 1 0 0 smtp
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982239] [18755] 108 18755 11107 159 1 0 0 smtp
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982242] [18759] 108 18759 11107 159 1 0 0 smtp
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982245] [18763] 108 18763 6792 69 1 0 0 scache
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982248] [20228] 1012 20228 819906 756597 0 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982251] [23539] 1012 23539 74092 11165 1 0 0 php
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982254] Out of memory: Kill process 20228 (php) score 789 or sacrifice child
Jul 16 02:15:06 ip-10-61-174-46 kernel: [1467801.982265] Killed process 20228 (php) total-vm:3279624kB, anon-rss:3026388kB, file-rss:0kB
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024273] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024277] php cpuset=/ mems_allowed=0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024280] Pid: 23894, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024282] Call Trace:
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024289] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024291] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024294] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024296] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024300] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024303] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024305] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024308] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024311] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024314] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024316] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024319] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024321] [<ffffffff811425ea>] ? find_vma+0x3a/0x70
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024325] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024327] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024329] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024332] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024335] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024336] Mem-Info:
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024337] Node 0 DMA per-cpu:
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024339] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024341] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024342] Node 0 DMA32 per-cpu:
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024344] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024345] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024347] active_anon:932255 inactive_anon:1393 isolated_anon:0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024348] active_file:0 inactive_file:9 isolated_file:0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024349] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024349] free:4038 slab_reclaimable:2831 slab_unreclaimable:2704
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024350] mapped:1353 shmem:1405 pagetables:3348 bounce:0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024352] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024358] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024361] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3729020kB inactive_anon:5572kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:11324kB slab_unreclaimable:10816kB kernel_stack:808kB pagetables:13392kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:106 all_unreclaimable? yes
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024368] lowmem_reserve[]: 0 0 0 0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024370] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024377] Node 0 DMA32: 959*4kB 244*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024383] 1430 total pagecache pages
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024385] 0 pages in swap cache
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024386] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024387] Free swap = 0kB
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.024388] Total swap = 0kB
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029852] 985072 pages RAM
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029855] 24998 pages reserved
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029856] 3790 pages shared
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029857] 953278 pages non-shared
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029859] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029867] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029871] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029874] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029877] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029881] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029884] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029887] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029891] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029895] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029898] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029901] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029904] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029907] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029910] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029913] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029916] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029919] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029923] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029926] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029929] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029932] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029935] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029938] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029942] [13468] 0 13468 89584 2501 1 0 0 php5-fpm
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029945] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029948] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029951] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029955] [ 9739] 108 9739 6792 76 0 0 0 pickup
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029958] [15986] 108 15986 6795 75 1 0 0 trivial-rewrite
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029961] [18742] 108 18742 6818 82 1 0 0 cleanup
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029965] [18743] 108 18743 11107 159 0 0 0 smtp
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029968] [18747] 108 18747 11107 159 0 0 0 smtp
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029971] [18751] 108 18751 11107 158 0 0 0 smtp
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029973] [18755] 108 18755 11107 159 0 0 0 smtp
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029976] [18759] 108 18759 11107 158 1 0 0 smtp
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029979] [18763] 108 18763 6792 69 1 0 0 scache
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029983] [23894] 1012 23894 818704 755477 0 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029986] [24899] 1012 24899 73914 11008 0 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029989] [25087] 1012 25087 1099 25 1 0 0 sh
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029991] [25088] 1012 25088 63799 1174 0 0 0 php
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.029994] Out of memory: Kill process 23894 (php) score 788 or sacrifice child
Jul 16 02:16:30 ip-10-61-174-46 kernel: [1467886.030005] Killed process 23894 (php) total-vm:3274816kB, anon-rss:3021908kB, file-rss:0kB
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822097] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822101] php cpuset=/ mems_allowed=0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822103] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822105] Call Trace:
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822112] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822115] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822117] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822120] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822123] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822128] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822131] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822133] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822136] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822140] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822142] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822144] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822147] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822149] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822153] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822156] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822159] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822162] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822164] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822167] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822170] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822171] Mem-Info:
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822172] Node 0 DMA per-cpu:
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822174] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822176] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822177] Node 0 DMA32 per-cpu:
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822179] CPU 0: hi: 186, btch: 31 usd: 68
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822180] CPU 1: hi: 186, btch: 31 usd: 81
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822182] active_anon:932002 inactive_anon:1393 isolated_anon:0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822183] active_file:39 inactive_file:17 isolated_file:19
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822184] unevictable:0 dirty:38 writeback:0 unstable:0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822184] free:4027 slab_reclaimable:2404 slab_unreclaimable:2720
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822185] mapped:1364 shmem:1405 pagetables:3822 bounce:0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822187] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822193] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822196] Node 0 DMA32 free:7772kB min:7848kB low:9808kB high:11772kB active_anon:3728008kB inactive_anon:5572kB active_file:156kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:152kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:9616kB slab_unreclaimable:10880kB kernel_stack:872kB pagetables:15288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:484 all_unreclaimable? yes
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822203] lowmem_reserve[]: 0 0 0 0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822205] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822212] Node 0 DMA32: 999*4kB 219*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822219] 1467 total pagecache pages
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822220] 0 pages in swap cache
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822221] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822222] Free swap = 0kB
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.822223] Total swap = 0kB
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827755] 985072 pages RAM
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827757] 24998 pages reserved
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827758] 3786 pages shared
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827760] 953196 pages non-shared
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827761] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827770] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827774] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827777] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827780] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827783] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827786] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827790] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827793] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827797] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827800] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827803] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827806] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827809] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827812] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827816] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827819] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827822] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827825] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827828] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827831] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827834] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827837] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827840] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827844] [13468] 0 13468 89584 2534 1 0 0 php5-fpm
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827847] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827850] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827853] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827856] [ 9739] 108 9739 6792 76 1 0 0 pickup
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827860] [25188] 1012 25188 793949 730745 0 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827863] [27331] 1012 27331 74722 11759 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827866] [27366] 108 27366 6818 74 1 0 0 cleanup
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827869] [27367] 108 27367 6795 70 1 0 0 trivial-rewrite
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827872] [27368] 108 27368 11107 158 1 0 0 smtp
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827875] [27372] 108 27372 11107 158 1 0 0 smtp
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827878] [27376] 108 27376 11107 158 1 0 0 smtp
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827881] [27380] 108 27380 6792 70 1 0 0 scache
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827884] [27384] 108 27384 11108 160 1 0 0 smtp
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827887] [27477] 1012 27477 1099 25 1 0 0 sh
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827890] [27479] 1012 27479 68825 5873 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827893] [27482] 1012 27482 1099 26 1 0 0 sh
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827896] [27483] 1012 27483 68825 5872 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827899] [27494] 1012 27494 1099 25 1 0 0 sh
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827902] [27495] 1012 27495 4738 65 1 0 0 git
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827905] [27497] 1012 27497 65898 3446 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827908] [27500] 1012 27500 1099 25 1 0 0 sh
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827911] [27501] 1012 27501 1099 25 1 0 0 sh
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827914] [27502] 1012 27502 65638 3196 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827917] [27503] 1012 27503 4736 65 1 0 0 git
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827920] [27504] 1012 27504 65637 3186 1 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827923] [27506] 1012 27506 1099 24 1 0 0 sh
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827926] [27507] 1012 27507 65639 3197 0 0 0 php
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827928] Out of memory: Kill process 25188 (php) score 762 or sacrifice child
Jul 16 02:19:10 ip-10-61-174-46 kernel: [1468045.827940] Killed process 25188 (php) total-vm:3175796kB, anon-rss:2922976kB, file-rss:4kB
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716122] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716126] php cpuset=/ mems_allowed=0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716129] Pid: 27605, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716131] Call Trace:
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716142] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716145] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716148] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716154] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716157] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716160] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716163] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716166] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716168] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716170] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716173] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716175] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716178] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716181] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716183] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716186] [<ffffffff811425e0>] ? find_vma+0x30/0x70
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716188] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716190] Mem-Info:
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716192] Node 0 DMA per-cpu:
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716194] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716195] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716196] Node 0 DMA32 per-cpu:
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716198] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716199] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716201] active_anon:932888 inactive_anon:1393 isolated_anon:0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716202] active_file:17 inactive_file:38 isolated_file:0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716203] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716203] free:4048 slab_reclaimable:2455 slab_unreclaimable:2692
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716204] mapped:1341 shmem:1405 pagetables:3227 bounce:0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716206] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716212] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716215] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731552kB inactive_anon:5572kB active_file:68kB inactive_file:152kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5364kB shmem:5620kB slab_reclaimable:9820kB slab_unreclaimable:10768kB kernel_stack:784kB pagetables:12908kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4266 all_unreclaimable? yes
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716222] lowmem_reserve[]: 0 0 0 0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716224] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716231] Node 0 DMA32: 1125*4kB 164*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7972kB
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716237] 1467 total pagecache pages
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716239] 0 pages in swap cache
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716240] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716241] Free swap = 0kB
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.716242] Total swap = 0kB
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721662] 985072 pages RAM
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721665] 24998 pages reserved
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721666] 3797 pages shared
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721668] 953304 pages non-shared
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721669] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721678] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721682] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721685] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721688] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721691] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721695] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721698] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721701] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721705] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721708] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721712] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721715] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721718] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721721] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721724] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721727] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721730] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721734] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721737] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721740] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721743] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721746] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721749] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721752] [13468] 0 13468 89584 2519 1 0 0 php5-fpm
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721755] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721759] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721762] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721765] [ 9739] 108 9739 6792 76 1 0 0 pickup
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721768] [27366] 108 27366 6818 74 1 0 0 cleanup
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721771] [27367] 108 27367 6795 70 0 0 0 trivial-rewrite
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721774] [27368] 108 27368 11107 158 1 0 0 smtp
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721777] [27372] 108 27372 11107 158 1 0 0 smtp
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721780] [27376] 108 27376 11107 158 1 0 0 smtp
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721783] [27380] 108 27380 6792 70 1 0 0 scache
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721786] [27384] 108 27384 11108 160 1 0 0 smtp
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721789] [27605] 1012 27605 826082 762859 0 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721792] [28069] 1012 28069 68580 5605 1 0 0 php
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721795] Out of memory: Kill process 27605 (php) score 796 or sacrifice child
Jul 16 02:20:05 ip-10-61-174-46 kernel: [1468100.721806] Killed process 27605 (php) total-vm:3304328kB, anon-rss:3051348kB, file-rss:88kB
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812125] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812130] php cpuset=/ mems_allowed=0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812133] Pid: 28423, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812135] Call Trace:
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812142] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812146] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812149] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812153] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812159] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812162] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812167] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812170] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812174] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812177] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812181] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812184] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812188] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812191] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812194] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812202] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812207] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812209] Mem-Info:
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812211] Node 0 DMA per-cpu:
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812216] Node 0 DMA32 per-cpu:
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812223] active_anon:932790 inactive_anon:1393 isolated_anon:0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812224] active_file:0 inactive_file:0 isolated_file:33
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812225] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812225] free:4048 slab_reclaimable:2351 slab_unreclaimable:2655
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812226] mapped:1347 shmem:1405 pagetables:3351 bounce:0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812237] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812241] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731160kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):132kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5620kB slab_reclaimable:9404kB slab_unreclaimable:10620kB kernel_stack:776kB pagetables:13404kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:33 all_unreclaimable? yes
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812251] lowmem_reserve[]: 0 0 0 0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812255] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812264] Node 0 DMA32: 750*4kB 335*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812273] 1465 total pagecache pages
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812275] 0 pages in swap cache
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812276] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812278] Free swap = 0kB
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.812279] Total swap = 0kB
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817814] 985072 pages RAM
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817816] 24998 pages reserved
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817817] 3784 pages shared
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817818] 953330 pages non-shared
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817820] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817829] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817832] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817836] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817839] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817842] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817845] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817849] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817852] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817856] [ 790] 101 790 63477 644 0 0 0 rsyslogd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817859] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817862] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817866] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817869] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817872] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817875] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817878] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817881] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817885] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817888] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817891] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817894] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817897] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817900] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817903] [13468] 0 13468 89584 2540 0 0 0 php5-fpm
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817906] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817909] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817913] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817916] [ 9739] 108 9739 6792 76 1 0 0 pickup
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817919] [27366] 108 27366 6818 82 1 0 0 cleanup
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817922] [27367] 108 27367 6795 74 1 0 0 trivial-rewrite
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817925] [27372] 108 27372 11107 157 1 0 0 smtp
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817928] [27376] 108 27376 11107 158 1 0 0 smtp
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817931] [27380] 108 27380 6792 73 0 0 0 scache
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817934] [27384] 108 27384 11108 160 1 0 0 smtp
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817937] [28423] 1012 28423 820390 757125 0 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817940] [29972] 1012 29972 73391 10450 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817944] [30186] 1012 30186 1099 24 0 0 0 sh
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817946] [30187] 1012 30187 1099 24 0 0 0 sh
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817949] [30188] 1012 30188 36197 439 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817952] [30189] 1012 30189 39759 458 1 0 0 php
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817955] Out of memory: Kill process 28423 (php) score 790 or sacrifice child
Jul 16 02:22:03 ip-10-61-174-46 kernel: [1468218.817967] Killed process 28423 (php) total-vm:3281560kB, anon-rss:3028500kB, file-rss:0kB
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570106] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570110] php cpuset=/ mems_allowed=0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570112] Pid: 30282, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570114] Call Trace:
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570121] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570123] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570126] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570129] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570132] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570135] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570137] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570140] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570143] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570146] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570149] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570151] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570154] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570156] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570160] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570162] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570164] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570167] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570169] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570171] Mem-Info:
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570172] Node 0 DMA per-cpu:
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570174] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570175] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570176] Node 0 DMA32 per-cpu:
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570178] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570179] CPU 1: hi: 186, btch: 31 usd: 26
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570182] active_anon:932394 inactive_anon:1393 isolated_anon:0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570183] active_file:0 inactive_file:4 isolated_file:0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570183] unevictable:0 dirty:23 writeback:0 unstable:0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570184] free:4030 slab_reclaimable:2515 slab_unreclaimable:2647
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570185] mapped:1356 shmem:1405 pagetables:3581 bounce:0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570186] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570193] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570196] Node 0 DMA32 free:7784kB min:7848kB low:9808kB high:11772kB active_anon:3729576kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:92kB writeback:0kB mapped:5424kB shmem:5620kB slab_reclaimable:10060kB slab_unreclaimable:10588kB kernel_stack:808kB pagetables:14324kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:62 all_unreclaimable? yes
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570202] lowmem_reserve[]: 0 0 0 0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570205] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570211] Node 0 DMA32: 948*4kB 229*8kB 5*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7784kB
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570218] 1438 total pagecache pages
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570219] 0 pages in swap cache
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570220] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570222] Free swap = 0kB
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.570222] Total swap = 0kB
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575727] 985072 pages RAM
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575730] 24998 pages reserved
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575731] 3778 pages shared
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575732] 953307 pages non-shared
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575734] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575743] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575746] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575749] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575752] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575755] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575759] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575762] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575765] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575769] [ 790] 101 790 63477 644 1 0 0 rsyslogd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575772] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575776] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575779] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575782] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575785] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575788] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575791] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575794] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575797] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575800] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575804] [ 654] 1012 654 70351 5770 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575807] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575810] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575813] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575816] [13468] 0 13468 89584 2525 0 0 0 php5-fpm
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575819] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575822] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575825] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575828] [30282] 1012 30282 803002 739760 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575831] [31292] 108 31292 6792 70 0 0 0 pickup
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575834] [ 1416] 1012 1416 68580 5606 1 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575837] [ 1508] 1012 1508 1099 24 1 0 0 sh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575840] [ 1509] 1012 1509 1099 24 0 0 0 sh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575843] [ 1510] 1012 1510 68825 5875 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575846] [ 1511] 1012 1511 68825 5874 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575849] [ 1522] 1012 1522 1099 26 0 0 0 sh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575852] [ 1523] 1012 1523 1099 25 0 0 0 sh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575855] [ 1524] 1012 1524 5216 69 0 0 0 git
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575858] [ 1525] 1012 1525 14040 63 0 0 0 git
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575861] [ 1526] 1012 1526 68499 5551 0 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575863] [ 1527] 1012 1527 68499 5552 1 0 0 php
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575866] [ 1530] 1012 1530 1099 24 0 0 0 sh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575869] [ 1531] 1012 1531 10901 165 0 0 0 ssh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575871] [ 1532] 1012 1532 1099 24 0 0 0 sh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575874] [ 1533] 1012 1533 10901 165 0 0 0 ssh
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575877] Out of memory: Kill process 30282 (php) score 772 or sacrifice child
Jul 16 02:26:01 ip-10-61-174-46 kernel: [1468456.575888] Killed process 30282 (php) total-vm:3212008kB, anon-rss:2959040kB, file-rss:0kB
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.115988] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.115991] ntpd cpuset=/ mems_allowed=0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.115994] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.115996] Call Trace:
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116003] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116006] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116008] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116011] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116014] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116018] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116020] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116023] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116025] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116029] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116031] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116034] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116037] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116040] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116044] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116045] Mem-Info:
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116046] Node 0 DMA per-cpu:
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116048] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116051] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116063] Node 0 DMA32 per-cpu:
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116066] CPU 0: hi: 186, btch: 31 usd: 52
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116068] CPU 1: hi: 186, btch: 31 usd: 37
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116071] active_anon:932382 inactive_anon:1393 isolated_anon:0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116072] active_file:14 inactive_file:4 isolated_file:28
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116073] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116074] free:4025 slab_reclaimable:2445 slab_unreclaimable:2684
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116075] mapped:1381 shmem:1405 pagetables:3574 bounce:0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116077] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116085] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116089] Node 0 DMA32 free:7764kB min:7848kB low:9808kB high:11772kB active_anon:3729528kB inactive_anon:5572kB active_file:56kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):112kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:9780kB slab_unreclaimable:10736kB kernel_stack:832kB pagetables:14296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:210 all_unreclaimable? yes
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116098] lowmem_reserve[]: 0 0 0 0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116101] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116112] Node 0 DMA32: 705*4kB 149*8kB 45*16kB 25*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7772kB
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116122] 1420 total pagecache pages
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116123] 0 pages in swap cache
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116125] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116127] Free swap = 0kB
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.116128] Total swap = 0kB
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121648] 985072 pages RAM
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121650] 24998 pages reserved
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121651] 3781 pages shared
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121652] 953275 pages non-shared
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121653] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121661] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121664] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121666] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121668] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121670] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121673] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121675] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121678] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121681] [ 790] 101 790 63477 644 0 0 0 rsyslogd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121683] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121685] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121688] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121690] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121692] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121694] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121696] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121699] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121701] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121703] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121706] [ 654] 1012 654 70351 5770 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121708] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121710] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121712] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121714] [13468] 0 13468 89584 2510 1 0 0 php5-fpm
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121716] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121718] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121721] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121723] [31292] 108 31292 6792 76 0 0 0 pickup
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121726] [ 1693] 1012 1693 803388 740117 1 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121728] [ 4955] 1012 4955 74590 11669 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121730] [ 5088] 108 5088 6818 73 0 0 0 cleanup
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121732] [ 5089] 108 5089 6795 70 1 0 0 trivial-rewrite
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121734] [ 5090] 108 5090 11107 158 0 0 0 smtp
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121736] [ 5094] 108 5094 11107 158 0 0 0 smtp
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121738] [ 5095] 1012 5095 1099 25 0 0 0 sh
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121741] [ 5096] 1012 5096 68825 5873 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121743] [ 5097] 1012 5097 1099 24 0 0 0 sh
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121745] [ 5098] 1012 5098 68825 5872 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121747] [ 5109] 1012 5109 1099 26 0 0 0 sh
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121749] [ 5110] 1012 5110 1099 25 0 0 0 sh
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121751] [ 5111] 1012 5111 4651 58 0 0 0 git
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121753] [ 5112] 1012 5112 4682 57 0 0 0 git
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121755] [ 5113] 1012 5113 64727 2255 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121757] [ 5114] 1012 5114 64727 2258 0 0 0 php
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121759] Out of memory: Kill process 1693 (php) score 772 or sacrifice child
Jul 16 02:29:56 ip-10-61-174-46 kernel: [1468692.121769] Killed process 1693 (php) total-vm:3213552kB, anon-rss:2960468kB, file-rss:0kB
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948130] git invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948134] git cpuset=/ mems_allowed=0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948136] Pid: 5785, comm: git Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948138] Call Trace:
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948145] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948148] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948153] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948156] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948159] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948162] [<ffffffff8113c7cf>] do_wp_page+0xcf/0x740
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948164] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948171] [<ffffffff8113ea2b>] handle_pte_fault+0x1cb/0x200
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948174] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948177] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948180] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948182] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948185] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948189] [<ffffffff8131b38d>] ? __put_user_4+0x1d/0x30
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948191] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948192] Mem-Info:
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948194] Node 0 DMA per-cpu:
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948196] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948197] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948198] Node 0 DMA32 per-cpu:
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948200] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948201] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948203] active_anon:932015 inactive_anon:1393 isolated_anon:0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948204] active_file:0 inactive_file:12 isolated_file:0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948205] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948205] free:4048 slab_reclaimable:2284 slab_unreclaimable:2726
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948206] mapped:1382 shmem:1405 pagetables:3889 bounce:0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948208] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948214] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948217] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728060kB inactive_anon:5572kB active_file:0kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5528kB shmem:5620kB slab_reclaimable:9136kB slab_unreclaimable:10904kB kernel_stack:936kB pagetables:15556kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9916 all_unreclaimable? yes
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948224] lowmem_reserve[]: 0 0 0 0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948226] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948233] Node 0 DMA32: 710*4kB 156*8kB 42*16kB 19*32kB 7*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7992kB
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948240] 1448 total pagecache pages
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948241] 0 pages in swap cache
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948242] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948243] Free swap = 0kB
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.948244] Total swap = 0kB
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953673] 985072 pages RAM
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953675] 24998 pages reserved
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953676] 3854 pages shared
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953678] 953224 pages non-shared
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953679] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953688] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953691] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953694] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953697] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953701] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953704] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953707] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953710] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953714] [ 790] 101 790 63477 644 0 0 0 rsyslogd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953717] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953721] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953724] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953727] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953730] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953733] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953736] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953739] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953742] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953746] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953749] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953752] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953755] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953758] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953761] [13468] 0 13468 89584 2486 0 0 0 php5-fpm
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953764] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953767] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953770] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953774] [31292] 108 31292 6792 76 0 0 0 pickup
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953777] [ 5088] 108 5088 6818 81 0 0 0 cleanup
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953780] [ 5089] 108 5089 6795 70 0 0 0 trivial-rewrite
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953783] [ 5090] 108 5090 11107 158 0 0 0 smtp
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953786] [ 5094] 108 5094 11107 158 0 0 0 smtp
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953789] [ 5222] 1012 5222 790496 727222 0 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953793] [ 5621] 1012 5621 68580 5606 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953796] [ 5746] 1012 5746 1099 25 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953799] [ 5747] 1012 5747 1099 24 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953802] [ 5748] 1012 5748 68825 5875 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953805] [ 5749] 1012 5749 68825 5870 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953807] [ 5750] 1012 5750 1099 25 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953810] [ 5751] 1012 5751 68825 5872 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953813] [ 5759] 1012 5759 1099 26 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953816] [ 5760] 1012 5760 14040 65 0 0 0 git
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953819] [ 5761] 1012 5761 68499 5551 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953822] [ 5767] 1012 5767 1099 26 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953825] [ 5768] 1012 5768 4658 64 0 0 0 git
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953828] [ 5769] 1012 5769 68499 5551 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953831] [ 5775] 1012 5775 1099 25 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953834] [ 5776] 1012 5776 4651 57 1 0 0 git
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953837] [ 5777] 1012 5777 68499 5552 1 0 0 php
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953840] [ 5779] 1012 5779 1099 24 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953843] [ 5780] 1012 5780 10901 176 0 0 0 ssh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953846] [ 5781] 1012 5781 1099 25 1 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953849] [ 5782] 1012 5782 10901 175 0 0 0 ssh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953852] [ 5783] 1012 5783 1099 24 0 0 0 sh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953855] [ 5784] 1012 5784 8240 109 0 0 0 ssh
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953857] [ 5785] 1012 5785 14040 65 1 0 0 git
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953860] Out of memory: Kill process 5222 (php) score 759 or sacrifice child
Jul 16 02:30:39 ip-10-61-174-46 kernel: [1468734.953871] Killed process 5222 (php) total-vm:3161984kB, anon-rss:2908888kB, file-rss:0kB
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916136] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916141] php5-fpm cpuset=/ mems_allowed=0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916144] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916146] Call Trace:
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916157] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916160] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916164] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916168] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916172] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916175] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916179] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916184] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916189] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916192] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916196] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916199] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916202] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916205] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916209] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916214] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916219] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916222] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916224] Mem-Info:
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916226] Node 0 DMA per-cpu:
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916229] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916230] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916232] Node 0 DMA32 per-cpu:
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916236] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916240] active_anon:933024 inactive_anon:1393 isolated_anon:0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916241] active_file:17 inactive_file:36 isolated_file:0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916241] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916242] free:4033 slab_reclaimable:2401 slab_unreclaimable:2576
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916243] mapped:1362 shmem:1405 pagetables:3260 bounce:0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916245] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916254] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916258] Node 0 DMA32 free:7796kB min:7848kB low:9808kB high:11772kB active_anon:3732096kB inactive_anon:5572kB active_file:68kB inactive_file:144kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:9604kB slab_unreclaimable:10304kB kernel_stack:760kB pagetables:13040kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:525 all_unreclaimable? yes
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916266] lowmem_reserve[]: 0 0 0 0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916270] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916278] Node 0 DMA32: 822*4kB 270*8kB 23*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7928kB
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916285] 1432 total pagecache pages
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916286] 0 pages in swap cache
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916287] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916288] Free swap = 0kB
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.916289] Total swap = 0kB
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921759] 985072 pages RAM
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921761] 24998 pages reserved
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921762] 3777 pages shared
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921763] 953339 pages non-shared
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921765] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921773] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921777] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921780] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921783] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921786] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921790] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921793] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921796] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921800] [ 790] 101 790 63477 644 0 0 0 rsyslogd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921803] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921806] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921810] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921813] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921816] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921819] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921822] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921825] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921828] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921831] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921834] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921837] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921841] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921844] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921847] [13468] 0 13468 89584 2528 0 0 0 php5-fpm
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921850] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921853] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921856] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921860] [31292] 108 31292 6792 76 1 0 0 pickup
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921863] [ 5901] 1012 5901 825887 762649 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921866] [ 7572] 1012 7572 68580 5605 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921869] [ 7805] 1012 7805 1099 25 1 0 0 sh
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921872] [ 7806] 1012 7806 1099 25 1 0 0 sh
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921875] [ 7807] 1012 7807 42681 543 0 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921878] [ 7808] 1012 7808 42681 542 1 0 0 php
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921881] Out of memory: Kill process 5901 (php) score 796 or sacrifice child
Jul 16 02:32:55 ip-10-61-174-46 kernel: [1468870.921892] Killed process 5901 (php) total-vm:3303548kB, anon-rss:3050596kB, file-rss:0kB
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213348] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213353] php cpuset=/ mems_allowed=0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213357] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213358] Call Trace:
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213365] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213368] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213370] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213373] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213376] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213379] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213381] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213384] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213387] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213391] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213393] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213396] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213399] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213402] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213406] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213409] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213412] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213413] Mem-Info:
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213414] Node 0 DMA per-cpu:
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213416] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213418] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213419] Node 0 DMA32 per-cpu:
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213420] CPU 0: hi: 186, btch: 31 usd: 57
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213422] CPU 1: hi: 186, btch: 31 usd: 40
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213424] active_anon:932489 inactive_anon:1393 isolated_anon:0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213425] active_file:20 inactive_file:17 isolated_file:10
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213426] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213426] free:4039 slab_reclaimable:2451 slab_unreclaimable:2643
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213427] mapped:1348 shmem:1405 pagetables:3549 bounce:0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213428] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213435] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213438] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3729956kB inactive_anon:5572kB active_file:80kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):40kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5392kB shmem:5620kB slab_reclaimable:9804kB slab_unreclaimable:10572kB kernel_stack:832kB pagetables:14196kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:168 all_unreclaimable? yes
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213445] lowmem_reserve[]: 0 0 0 0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213447] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213454] Node 0 DMA32: 737*4kB 343*8kB 5*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213460] 1441 total pagecache pages
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213462] 0 pages in swap cache
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213463] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213464] Free swap = 0kB
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.213465] Total swap = 0kB
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218849] 985072 pages RAM
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218852] 24998 pages reserved
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218853] 3786 pages shared
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218854] 953251 pages non-shared
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218856] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218865] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218869] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218872] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218875] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218878] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218881] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218885] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218888] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218892] [ 790] 101 790 63477 644 0 0 0 rsyslogd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218895] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218898] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218901] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218904] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218907] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218911] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218914] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218917] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218920] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218923] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218926] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218929] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218932] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218935] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218938] [13468] 0 13468 89584 2495 0 0 0 php5-fpm
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218942] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218945] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218948] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218951] [31292] 108 31292 6792 76 0 0 0 pickup
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218954] [ 7871] 108 7871 6818 73 1 0 0 cleanup
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218958] [ 7872] 108 7872 6795 69 0 0 0 trivial-rewrite
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218960] [ 7873] 108 7873 11107 158 0 0 0 smtp
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218964] [ 7877] 108 7877 11107 157 1 0 0 smtp
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218967] [ 7900] 1012 7900 813143 749870 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218970] [ 9957] 1012 9957 68580 5605 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218973] [10076] 1012 10076 1099 25 0 0 0 sh
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218976] [10077] 1012 10077 68825 5874 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218979] [10078] 1012 10078 1099 25 0 0 0 sh
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218982] [10079] 1012 10079 68825 5872 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218984] [10090] 1012 10090 1099 26 0 0 0 sh
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218987] [10091] 1012 10091 1099 26 0 0 0 sh
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218990] [10092] 1012 10092 4651 58 0 0 0 git
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218993] [10093] 1012 10093 14040 62 0 0 0 git
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218997] [10094] 1012 10094 40280 493 1 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.218999] [10095] 1012 10095 39759 480 0 0 0 php
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.219002] Out of memory: Kill process 7900 (php) score 782 or sacrifice child
Jul 16 02:35:29 ip-10-61-174-46 kernel: [1469025.219013] Killed process 7900 (php) total-vm:3252572kB, anon-rss:2999480kB, file-rss:0kB
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028136] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028140] php cpuset=/ mems_allowed=0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028142] Pid: 14392, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028144] Call Trace:
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028150] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028153] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028155] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028158] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028161] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028164] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028166] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028169] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028172] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028176] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028178] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028180] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028183] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028185] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028188] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028191] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028193] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028196] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028197] Mem-Info:
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028198] Node 0 DMA per-cpu:
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028200] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028202] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028203] Node 0 DMA32 per-cpu:
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028204] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028206] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028208] active_anon:932469 inactive_anon:1393 isolated_anon:0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028209] active_file:8 inactive_file:8 isolated_file:0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028210] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028210] free:4048 slab_reclaimable:3066 slab_unreclaimable:2630
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028211] mapped:1367 shmem:1405 pagetables:3075 bounce:0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028212] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028219] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028222] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729876kB inactive_anon:5572kB active_file:32kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5620kB slab_reclaimable:12264kB slab_unreclaimable:10520kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9048 all_unreclaimable? yes
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028228] lowmem_reserve[]: 0 0 0 0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028231] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028238] Node 0 DMA32: 1466*4kB 9*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8032kB
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028244] 1466 total pagecache pages
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028245] 0 pages in swap cache
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028246] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028248] Free swap = 0kB
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.028248] Total swap = 0kB
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033567] 985072 pages RAM
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033569] 24998 pages reserved
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033570] 3808 pages shared
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033571] 953285 pages non-shared
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033573] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033582] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033586] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033589] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033592] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033595] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033598] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033601] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033605] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033609] [ 790] 101 790 63477 644 0 0 0 rsyslogd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033612] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033615] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033618] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033621] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033625] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033628] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033631] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033634] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033637] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033640] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033643] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033646] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033649] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033652] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033656] [13468] 0 13468 89584 2506 0 0 0 php5-fpm
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033659] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033661] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033665] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033668] [31292] 108 31292 6792 76 0 0 0 pickup
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033671] [14392] 1012 14392 826544 763298 0 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033674] [21143] 1012 21143 68580 5604 1 0 0 php
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033677] Out of memory: Kill process 14392 (php) score 796 or sacrifice child
Jul 16 02:47:47 ip-10-61-174-46 kernel: [1469763.033689] Killed process 14392 (php) total-vm:3306176kB, anon-rss:3053192kB, file-rss:0kB
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388172] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388176] php cpuset=/ mems_allowed=0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388179] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388182] Call Trace:
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388190] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388193] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388196] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388199] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388204] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388208] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388211] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388215] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388219] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388222] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388226] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388230] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388233] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388238] [<ffffffff8164502b>] ? mm_fault_error+0x1/0xf3
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388241] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388246] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388249] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388252] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388256] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388258] Mem-Info:
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388260] Node 0 DMA per-cpu:
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388262] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388264] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388266] Node 0 DMA32 per-cpu:
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388267] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388269] CPU 1: hi: 186, btch: 31 usd: 18
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388271] active_anon:932476 inactive_anon:1393 isolated_anon:0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388272] active_file:0 inactive_file:0 isolated_file:0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388273] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388273] free:4048 slab_reclaimable:2323 slab_unreclaimable:2685
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388274] mapped:1351 shmem:1405 pagetables:3620 bounce:0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388276] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388282] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388285] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729904kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5404kB shmem:5620kB slab_reclaimable:9292kB slab_unreclaimable:10740kB kernel_stack:856kB pagetables:14480kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:25 all_unreclaimable? yes
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388291] lowmem_reserve[]: 0 0 0 0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388294] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388301] Node 0 DMA32: 750*4kB 210*8kB 34*16kB 15*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7944kB
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388307] 1463 total pagecache pages
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388308] 0 pages in swap cache
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388310] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388311] Free swap = 0kB
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.388312] Total swap = 0kB
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393778] 985072 pages RAM
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393780] 24998 pages reserved
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393781] 3816 pages shared
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393783] 953265 pages non-shared
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393785] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393793] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393797] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393800] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393803] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393806] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393809] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393813] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393816] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393820] [ 790] 101 790 63477 645 0 0 0 rsyslogd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393823] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393826] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393830] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393833] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393836] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393839] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393842] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393845] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393848] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393851] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393854] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393857] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393860] [ 658] 1012 658 200573 135992 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393863] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393867] [13468] 0 13468 89584 2522 0 0 0 php5-fpm
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393870] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393873] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393876] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393879] [31292] 108 31292 6792 76 1 0 0 pickup
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393883] [21324] 1012 21324 794039 730800 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393886] [21366] 1012 21366 79940 16959 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393889] [21501] 108 21501 6818 73 1 0 0 cleanup
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393892] [21503] 108 21503 6795 70 1 0 0 trivial-rewrite
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393895] [21504] 108 21504 6818 74 1 0 0 cleanup
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393898] [21505] 108 21505 11107 158 0 0 0 smtp
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393901] [21506] 108 21506 11107 157 0 0 0 smtp
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393904] [21510] 108 21510 11107 159 0 0 0 smtp
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393907] [21970] 1012 21970 1099 25 0 0 0 sh
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393910] [21971] 1012 21971 1099 24 0 0 0 sh
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393913] [21972] 1012 21972 68825 5869 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393916] [21973] 1012 21973 68825 5874 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393919] [21984] 1012 21984 1099 26 0 0 0 sh
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393921] [21985] 1012 21985 4682 58 0 0 0 git
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393924] [21986] 1012 21986 1099 26 0 0 0 sh
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393927] [21987] 1012 21987 66678 4229 0 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393930] [21988] 1012 21988 14040 62 0 0 0 git
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393933] [21989] 1012 21989 66677 4230 1 0 0 php
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393936] Out of memory: Kill process 21324 (php) score 762 or sacrifice child
Jul 16 02:48:30 ip-10-61-174-46 kernel: [1469806.393947] Killed process 21324 (php) total-vm:3176156kB, anon-rss:2923200kB, file-rss:0kB
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700109] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700113] php cpuset=/ mems_allowed=0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700116] Pid: 21366, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700117] Call Trace:
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700124] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700127] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700129] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700132] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700135] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700138] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700141] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700144] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700147] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700149] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700152] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700154] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700158] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700160] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700162] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700165] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700168] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700169] Mem-Info:
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700171] Node 0 DMA per-cpu:
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700172] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700174] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700175] Node 0 DMA32 per-cpu:
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700177] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700178] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700180] active_anon:932875 inactive_anon:1393 isolated_anon:0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700181] active_file:16 inactive_file:17 isolated_file:0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700182] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700182] free:4048 slab_reclaimable:2449 slab_unreclaimable:2673
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700183] mapped:1347 shmem:1405 pagetables:3202 bounce:0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700184] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700191] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700194] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731500kB inactive_anon:5572kB active_file:64kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5620kB slab_reclaimable:9796kB slab_unreclaimable:10692kB kernel_stack:776kB pagetables:12808kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4770 all_unreclaimable? yes
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700201] lowmem_reserve[]: 0 0 0 0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700203] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700210] Node 0 DMA32: 748*4kB 239*8kB 53*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7928kB
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700217] 1461 total pagecache pages
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700218] 0 pages in swap cache
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700219] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700220] Free swap = 0kB
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.700221] Total swap = 0kB
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705728] 985072 pages RAM
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705730] 24998 pages reserved
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705732] 3807 pages shared
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705733] 953306 pages non-shared
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705734] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705743] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705746] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705750] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705753] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705756] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705760] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705763] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705766] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705770] [ 790] 101 790 63477 645 0 0 0 rsyslogd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705774] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705777] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705780] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705783] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705786] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705790] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705793] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705796] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705799] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705802] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705805] [ 654] 1012 654 70351 5769 1 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705808] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705811] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705814] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705817] [13468] 0 13468 89584 2507 1 0 0 php5-fpm
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705820] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705823] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705827] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705830] [31292] 108 31292 6792 76 1 0 0 pickup
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705833] [21366] 1012 21366 826280 762980 0 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705836] [21501] 108 21501 6818 81 0 0 0 cleanup
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705839] [21503] 108 21503 6795 75 1 0 0 trivial-rewrite
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705843] [21504] 108 21504 6818 82 1 0 0 cleanup
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705846] [21505] 108 21505 11107 157 0 0 0 smtp
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705849] [21506] 108 21506 11107 156 1 0 0 smtp
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705852] [21510] 108 21510 11107 158 0 0 0 smtp
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705855] [22393] 1012 22393 68580 5604 1 0 0 php
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705858] Out of memory: Kill process 21366 (php) score 796 or sacrifice child
Jul 16 02:48:59 ip-10-61-174-46 kernel: [1469834.705870] Killed process 21366 (php) total-vm:3305120kB, anon-rss:3051920kB, file-rss:0kB
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364164] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364168] php cpuset=/ mems_allowed=0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364170] Pid: 23309, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364172] Call Trace:
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364179] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364181] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364183] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364186] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364191] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364194] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364198] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364200] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364203] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364207] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364209] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364212] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364215] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364217] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364220] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364222] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364225] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364227] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364229] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364231] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364234] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364235] Mem-Info:
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364236] Node 0 DMA per-cpu:
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364238] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364240] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364241] Node 0 DMA32 per-cpu:
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364242] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364244] CPU 1: hi: 186, btch: 31 usd: 8
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364246] active_anon:932390 inactive_anon:1393 isolated_anon:0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364247] active_file:0 inactive_file:8 isolated_file:0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364248] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364248] free:4040 slab_reclaimable:2616 slab_unreclaimable:2943
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364249] mapped:1353 shmem:1405 pagetables:3254 bounce:0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364250] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364257] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364260] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3729560kB inactive_anon:5572kB active_file:0kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10464kB slab_unreclaimable:11772kB kernel_stack:768kB pagetables:13016kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:26 all_unreclaimable? yes
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364266] lowmem_reserve[]: 0 0 0 0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364269] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364275] Node 0 DMA32: 1451*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364282] 1423 total pagecache pages
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364283] 0 pages in swap cache
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364284] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364285] Free swap = 0kB
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.364286] Total swap = 0kB
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369680] 985072 pages RAM
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369683] 24998 pages reserved
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369684] 3785 pages shared
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369685] 953310 pages non-shared
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369687] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369696] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369699] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369703] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369706] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369709] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369712] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369715] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369719] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369722] [ 790] 101 790 63477 645 0 0 0 rsyslogd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369726] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369729] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369732] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369735] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369738] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369741] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369744] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369747] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369751] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369754] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369757] [ 654] 1012 654 70351 5769 0 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369760] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369763] [ 658] 1012 658 200573 135992 1 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369766] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369770] [13468] 0 13468 89584 2492 0 0 0 php5-fpm
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369773] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369776] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369779] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369783] [22486] 1012 22486 78242 15247 0 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369786] [23309] 1012 23309 80746 17705 0 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369789] [31125] 1012 31125 798818 735418 1 0 0 php
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369792] [32026] 108 32026 6818 74 0 0 0 cleanup
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369795] [32027] 108 32027 6795 71 1 0 0 trivial-rewrite
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369798] [32028] 108 32028 11107 158 1 0 0 smtp
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369801] [32032] 108 32032 6792 71 1 0 0 pickup
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369804] [32033] 108 32033 11107 159 0 0 0 smtp
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369807] Out of memory: Kill process 31125 (php) score 767 or sacrifice child
Jul 16 03:01:28 ip-10-61-174-46 kernel: [1470584.369818] Killed process 31125 (php) total-vm:3195272kB, anon-rss:2941672kB, file-rss:0kB
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092137] sh invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092142] sh cpuset=/ mems_allowed=0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092145] Pid: 29357, comm: sh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092147] Call Trace:
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092155] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092158] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092161] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092164] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092168] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092172] [<ffffffff811894c3>] ? user_path_at_empty+0x63/0xa0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092174] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092177] [<ffffffff81196e8e>] ? vfsmount_lock_local_unlock+0x1e/0x30
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092179] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092181] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092185] [<ffffffff8117e6aa>] ? sys_newstat+0x2a/0x40
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092188] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092192] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092195] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092197] Mem-Info:
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092198] Node 0 DMA per-cpu:
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092200] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092201] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092202] Node 0 DMA32 per-cpu:
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092204] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092205] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092207] active_anon:932026 inactive_anon:1393 isolated_anon:0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092208] active_file:0 inactive_file:0 isolated_file:0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092209] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092210] free:4048 slab_reclaimable:2891 slab_unreclaimable:2869
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092210] mapped:1330 shmem:1405 pagetables:3390 bounce:0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092212] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092219] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092222] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728104kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5320kB shmem:5620kB slab_reclaimable:11564kB slab_unreclaimable:11476kB kernel_stack:784kB pagetables:13560kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092228] lowmem_reserve[]: 0 0 0 0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092230] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092237] Node 0 DMA32: 1467*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7964kB
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092244] 1451 total pagecache pages
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092245] 0 pages in swap cache
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092246] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092247] Free swap = 0kB
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.092248] Total swap = 0kB
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097807] 985072 pages RAM
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097810] 24998 pages reserved
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097811] 3804 pages shared
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097812] 953300 pages non-shared
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097814] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097822] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097826] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097829] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097832] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097835] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097839] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097842] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097845] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097849] [ 790] 101 790 63477 645 1 0 0 rsyslogd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097853] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097855] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097859] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097862] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097865] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097868] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097871] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097874] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097877] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097880] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097883] [ 654] 1012 654 70351 5770 0 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097886] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097889] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097892] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097895] [13468] 0 13468 89584 2511 0 0 0 php5-fpm
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097898] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097901] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097904] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097908] [22486] 1012 22486 80515 17576 0 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097911] [23309] 1012 23309 787252 723833 0 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097914] [ 2519] 1012 2519 79648 16622 0 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097917] [23550] 108 23550 6792 69 0 0 0 pickup
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097920] [29346] 1012 29346 1099 24 1 0 0 sh
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097923] [29347] 1012 29347 68825 5868 0 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097926] [29353] 1012 29353 1099 26 1 0 0 sh
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097929] [29354] 1012 29354 4736 65 1 0 0 git
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097932] [29355] 1012 29355 68499 5552 1 0 0 php
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097935] [29357] 1012 29357 1099 25 1 0 0 sh
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097937] Out of memory: Kill process 23309 (php) score 755 or sacrifice child
Jul 16 03:30:15 ip-10-61-174-46 kernel: [1472311.097949] Killed process 23309 (php) total-vm:3149008kB, anon-rss:2895320kB, file-rss:12kB
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462730] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462735] php cpuset=/ mems_allowed=0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462738] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462740] Call Trace:
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462748] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462752] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462755] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462759] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462763] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462768] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462771] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462775] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462780] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462784] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462787] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462791] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462793] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462795] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462798] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462801] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462805] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462808] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462810] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462812] Mem-Info:
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462813] Node 0 DMA per-cpu:
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462815] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462817] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462818] Node 0 DMA32 per-cpu:
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462819] CPU 0: hi: 186, btch: 31 usd: 39
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462821] CPU 1: hi: 186, btch: 31 usd: 63
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462823] active_anon:932694 inactive_anon:1393 isolated_anon:0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462824] active_file:20 inactive_file:4 isolated_file:15
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462825] unevictable:0 dirty:15 writeback:0 unstable:0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462825] free:4019 slab_reclaimable:2347 slab_unreclaimable:2847
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462826] mapped:1383 shmem:1405 pagetables:3256 bounce:0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462828] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462834] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462837] Node 0 DMA32 free:7740kB min:7848kB low:9808kB high:11772kB active_anon:3730776kB inactive_anon:5572kB active_file:80kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):60kB present:3862656kB mlocked:0kB dirty:60kB writeback:0kB mapped:5532kB shmem:5620kB slab_reclaimable:9388kB slab_unreclaimable:11388kB kernel_stack:752kB pagetables:13024kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:282 all_unreclaimable? yes
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462844] lowmem_reserve[]: 0 0 0 0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462846] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462853] Node 0 DMA32: 891*4kB 246*8kB 8*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7740kB
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462860] 1439 total pagecache pages
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462861] 0 pages in swap cache
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462862] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462863] Free swap = 0kB
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.462864] Total swap = 0kB
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468391] 985072 pages RAM
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468393] 24998 pages reserved
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468394] 3788 pages shared
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468395] 953273 pages non-shared
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468397] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468406] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468410] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468413] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468416] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468419] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468423] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468426] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468429] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468433] [ 790] 101 790 63477 645 0 0 0 rsyslogd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468436] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468440] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468443] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468446] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468449] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468452] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468455] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468459] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468462] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468465] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468468] [ 654] 1012 654 70351 5771 1 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468471] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468475] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468478] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468481] [13468] 0 13468 89584 2496 1 0 0 php5-fpm
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468484] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468487] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468490] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468493] [22486] 1012 22486 80515 17578 0 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468496] [ 2519] 1012 2519 79648 16625 1 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468500] [23550] 108 23550 6792 75 0 0 0 pickup
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468503] [29451] 1012 29451 793361 730117 0 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468506] [32501] 1012 32501 1099 25 1 0 0 sh
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468509] [32502] 1012 32502 68825 5871 0 0 0 php
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468512] [32508] 1012 32508 1099 26 1 0 0 sh
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468515] Out of memory: Kill process 29451 (php) score 762 or sacrifice child
Jul 16 03:33:44 ip-10-61-174-46 kernel: [1472519.468526] Killed process 29451 (php) total-vm:3173444kB, anon-rss:2920456kB, file-rss:12kB
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493424] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493428] php cpuset=/ mems_allowed=0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493430] Pid: 22486, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493432] Call Trace:
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493439] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493441] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493443] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493446] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493449] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493453] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493455] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493458] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493461] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493464] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493466] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493468] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493471] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493475] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493478] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493481] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493485] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493487] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493490] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493493] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493495] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493497] Mem-Info:
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493498] Node 0 DMA per-cpu:
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493500] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493502] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493503] Node 0 DMA32 per-cpu:
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493505] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493506] CPU 1: hi: 186, btch: 31 usd: 48
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493508] active_anon:932628 inactive_anon:1393 isolated_anon:0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493509] active_file:9 inactive_file:3 isolated_file:0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493510] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493510] free:4034 slab_reclaimable:2474 slab_unreclaimable:2823
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493511] mapped:1342 shmem:1405 pagetables:3276 bounce:0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493513] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493519] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493522] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3730512kB inactive_anon:5572kB active_file:36kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5368kB shmem:5620kB slab_reclaimable:9896kB slab_unreclaimable:11292kB kernel_stack:776kB pagetables:13104kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:280 all_unreclaimable? yes
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493529] lowmem_reserve[]: 0 0 0 0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493531] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493538] Node 0 DMA32: 836*4kB 241*8kB 28*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493545] 1432 total pagecache pages
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493546] 0 pages in swap cache
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493547] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493548] Free swap = 0kB
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.493549] Total swap = 0kB
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498955] 985072 pages RAM
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498957] 24998 pages reserved
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498959] 3779 pages shared
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498960] 953288 pages non-shared
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498962] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498970] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498974] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498977] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498981] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498984] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498987] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498990] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498994] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.498998] [ 790] 101 790 63477 645 1 0 0 rsyslogd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499001] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499004] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499008] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499010] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499014] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499017] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499020] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499023] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499026] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499029] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499032] [ 654] 1012 654 70351 5770 0 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499035] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499038] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499041] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499045] [13468] 0 13468 89584 2519 0 0 0 php5-fpm
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499048] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499051] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499054] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499057] [22486] 1012 22486 80515 17579 0 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499060] [ 2519] 1012 2519 79648 16625 0 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499063] [23550] 108 23550 6792 75 0 0 0 pickup
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499067] [32623] 1012 32623 798601 735379 1 0 0 php
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499070] [32630] 108 32630 6818 73 0 0 0 cleanup
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499073] [32632] 108 32632 6795 70 1 0 0 trivial-rewrite
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499076] [32633] 108 32633 11107 159 0 0 0 smtp
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499079] [32634] 108 32634 11107 159 1 0 0 smtp
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499082] [32638] 108 32638 11107 157 1 0 0 smtp
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499085] Out of memory: Kill process 32623 (php) score 767 or sacrifice child
Jul 16 03:35:18 ip-10-61-174-46 kernel: [1472614.499098] Killed process 32623 (php) total-vm:3194404kB, anon-rss:2941516kB, file-rss:0kB
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068131] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068135] php cpuset=/ mems_allowed=0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068137] Pid: 2519, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068139] Call Trace:
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068146] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068148] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068153] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068156] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068160] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068162] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068166] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068168] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068171] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068174] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068177] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068180] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068182] Mem-Info:
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068183] Node 0 DMA per-cpu:
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068185] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068186] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068187] Node 0 DMA32 per-cpu:
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068189] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068190] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068193] active_anon:931826 inactive_anon:1393 isolated_anon:0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068193] active_file:0 inactive_file:0 isolated_file:0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068194] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068195] free:4048 slab_reclaimable:2553 slab_unreclaimable:2950
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068195] mapped:1381 shmem:1405 pagetables:3758 bounce:0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068197] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068203] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068206] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727304kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:10212kB slab_unreclaimable:11800kB kernel_stack:888kB pagetables:15032kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068213] lowmem_reserve[]: 0 0 0 0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068215] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068222] Node 0 DMA32: 1137*4kB 139*8kB 5*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7788kB
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068229] 1478 total pagecache pages
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068230] 0 pages in swap cache
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068231] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068232] Free swap = 0kB
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.068233] Total swap = 0kB
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073755] 985072 pages RAM
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073757] 24998 pages reserved
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073759] 3808 pages shared
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073760] 953327 pages non-shared
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073762] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073770] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073773] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073777] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073780] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073783] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073786] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073790] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073793] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073797] [ 790] 101 790 63477 645 1 0 0 rsyslogd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073800] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073803] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073806] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073809] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073813] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073816] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073819] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073822] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073825] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073828] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073831] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073835] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073838] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073841] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073844] [13468] 0 13468 89584 2504 1 0 0 php5-fpm
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073847] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073850] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073853] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073856] [22486] 1012 22486 80515 17578 1 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073859] [ 2519] 1012 2519 777882 714526 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073862] [23550] 108 23550 6792 75 1 0 0 pickup
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073866] [ 1594] 1012 1594 76115 13215 1 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073869] [ 4651] 108 4651 6818 73 1 0 0 cleanup
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073872] [ 4652] 108 4652 6795 71 1 0 0 trivial-rewrite
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073875] [ 4653] 108 4653 11107 159 1 0 0 smtp
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073878] [ 4663] 108 4663 11107 158 1 0 0 smtp
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073881] [ 6918] 108 6918 11107 159 1 0 0 smtp
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073884] [ 8825] 1012 8825 1099 24 0 0 0 sh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073887] [ 8826] 1012 8826 68825 5871 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073890] [ 8827] 1012 8827 1099 24 1 0 0 sh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073893] [ 8828] 1012 8828 68825 5869 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073896] [ 8835] 1012 8835 1099 26 0 0 0 sh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073899] [ 8836] 1012 8836 5216 68 0 0 0 git
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073902] [ 8838] 1012 8838 68499 5552 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073905] [ 8843] 1012 8843 1099 25 0 0 0 sh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073908] [ 8844] 1012 8844 4669 58 0 0 0 git
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073911] [ 8845] 1012 8845 68499 5552 0 0 0 php
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073914] [ 8847] 1012 8847 1099 25 0 0 0 sh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073917] [ 8848] 1012 8848 1099 25 0 0 0 sh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073920] [ 8849] 1012 8849 10901 165 0 0 0 ssh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073923] [ 8850] 1012 8850 10901 165 1 0 0 ssh
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073925] Out of memory: Kill process 2519 (php) score 745 or sacrifice child
Jul 16 03:43:11 ip-10-61-174-46 kernel: [1473087.073937] Killed process 2519 (php) total-vm:3111528kB, anon-rss:2858012kB, file-rss:92kB
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952117] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952121] php cpuset=/ mems_allowed=0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952124] Pid: 22486, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952125] Call Trace:
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952133] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952135] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952137] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952140] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952144] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952147] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952149] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952152] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952157] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952159] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952162] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952165] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952167] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952170] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952172] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952175] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952177] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952179] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952181] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952184] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952186] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952188] Mem-Info:
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952189] Node 0 DMA per-cpu:
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952191] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952192] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952193] Node 0 DMA32 per-cpu:
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952195] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952196] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952199] active_anon:932411 inactive_anon:1393 isolated_anon:0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952200] active_file:0 inactive_file:1 isolated_file:0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952200] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952201] free:4048 slab_reclaimable:2934 slab_unreclaimable:2687
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952202] mapped:1355 shmem:1405 pagetables:3223 bounce:0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952203] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952210] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952213] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729644kB inactive_anon:5572kB active_file:0kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:11736kB slab_unreclaimable:10748kB kernel_stack:784kB pagetables:12892kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:505 all_unreclaimable? yes
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952219] lowmem_reserve[]: 0 0 0 0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952222] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952228] Node 0 DMA32: 1447*4kB 13*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7988kB
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952235] 1466 total pagecache pages
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952236] 0 pages in swap cache
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952238] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952239] Free swap = 0kB
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.952239] Total swap = 0kB
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957681] 985072 pages RAM
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957683] 24998 pages reserved
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957684] 3782 pages shared
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957685] 953315 pages non-shared
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957687] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957696] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957700] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957703] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957706] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957709] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957712] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957716] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957719] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957723] [ 790] 101 790 63477 646 0 0 0 rsyslogd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957727] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957730] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957733] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957736] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957739] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957742] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957745] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957748] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957751] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957755] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957758] [ 654] 1012 654 70351 5771 1 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957761] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957764] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957767] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957770] [13468] 0 13468 89584 2489 1 0 0 php5-fpm
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957773] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957776] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957780] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957783] [22486] 1012 22486 826816 763549 0 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957786] [11840] 108 11840 6818 73 0 0 0 cleanup
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957790] [11841] 108 11841 6795 70 0 0 0 trivial-rewrite
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957793] [12098] 108 12098 11108 160 0 0 0 smtp
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957796] [13715] 108 13715 11107 159 1 0 0 smtp
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957799] [13731] 108 13731 11107 158 1 0 0 smtp
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957802] [15621] 108 15621 11107 160 0 0 0 smtp
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957805] [15779] 108 15779 6792 70 0 0 0 pickup
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957808] [16751] 108 16751 6792 69 1 0 0 scache
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957811] [17329] 1012 17329 68580 5604 1 0 0 php
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957814] Out of memory: Kill process 22486 (php) score 796 or sacrifice child
Jul 16 03:52:36 ip-10-61-174-46 kernel: [1473651.957824] Killed process 22486 (php) total-vm:3307264kB, anon-rss:3054196kB, file-rss:0kB
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091815] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091818] php5-fpm cpuset=/ mems_allowed=0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091822] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091823] Call Trace:
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091830] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091833] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091835] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091838] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091841] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091845] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091847] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091850] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091853] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091856] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091859] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091861] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091864] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091867] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091870] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091874] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091878] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091881] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091883] Mem-Info:
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091884] Node 0 DMA per-cpu:
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091886] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091887] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091888] Node 0 DMA32 per-cpu:
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091890] CPU 0: hi: 186, btch: 31 usd: 47
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091891] CPU 1: hi: 186, btch: 31 usd: 110
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091894] active_anon:932315 inactive_anon:1393 isolated_anon:0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091894] active_file:12 inactive_file:0 isolated_file:0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091895] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091896] free:4025 slab_reclaimable:2472 slab_unreclaimable:2709
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091896] mapped:1361 shmem:1405 pagetables:3482 bounce:0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091898] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091904] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091907] Node 0 DMA32 free:7764kB min:7848kB low:9808kB high:11772kB active_anon:3729260kB inactive_anon:5572kB active_file:48kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:9888kB slab_unreclaimable:10836kB kernel_stack:848kB pagetables:13928kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:295 all_unreclaimable? yes
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091914] lowmem_reserve[]: 0 0 0 0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091916] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091923] Node 0 DMA32: 871*4kB 233*8kB 23*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7764kB
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091929] 1421 total pagecache pages
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091931] 0 pages in swap cache
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091932] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091933] Free swap = 0kB
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.091934] Total swap = 0kB
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097345] 985072 pages RAM
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097348] 24998 pages reserved
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097349] 3779 pages shared
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097350] 953209 pages non-shared
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097352] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097361] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097365] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097368] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097371] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097374] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097377] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097380] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097384] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097388] [ 790] 101 790 63477 646 1 0 0 rsyslogd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097391] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097394] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097397] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097400] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097404] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097407] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097410] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097413] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097416] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097419] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097422] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097426] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097429] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097432] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097435] [13468] 0 13468 89584 2528 1 0 0 php5-fpm
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097438] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097441] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097444] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097448] [11840] 108 11840 6818 73 0 0 0 cleanup
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097451] [11841] 108 11841 6795 70 1 0 0 trivial-rewrite
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097454] [12098] 108 12098 11108 160 0 0 0 smtp
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097457] [13715] 108 13715 11107 159 1 0 0 smtp
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097460] [13731] 108 13731 11107 158 1 0 0 smtp
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097463] [15621] 108 15621 11107 160 0 0 0 smtp
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097467] [15779] 108 15779 6792 70 0 0 0 pickup
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097470] [16751] 108 16751 6792 69 1 0 0 scache
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097473] [17682] 1012 17682 810563 747329 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097476] [18422] 1012 18422 72878 9906 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097479] [18607] 1012 18607 1099 24 1 0 0 sh
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097482] [18608] 1012 18608 68825 5871 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097485] [18614] 1012 18614 1099 25 1 0 0 sh
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097488] [18615] 1012 18615 4652 64 0 0 0 git
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097491] [18616] 1012 18616 68499 5552 1 0 0 php
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097495] [18618] 1012 18618 1099 24 1 0 0 sh
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097498] [18619] 1012 18619 10901 165 1 0 0 ssh
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097501] [18621] 1012 18621 4827 73 0 0 0 git
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097504] Out of memory: Kill process 17682 (php) score 780 or sacrifice child
Jul 16 03:53:42 ip-10-61-174-46 kernel: [1473718.097515] Killed process 17682 (php) total-vm:3242252kB, anon-rss:2989316kB, file-rss:0kB
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092865] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092868] php cpuset=/ mems_allowed=0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092871] Pid: 18758, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092873] Call Trace:
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092879] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092882] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092884] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092887] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092890] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092893] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092895] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092898] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092901] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092904] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092907] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092910] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092912] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092916] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092918] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092920] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092923] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092926] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092928] Mem-Info:
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092929] Node 0 DMA per-cpu:
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092931] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092932] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092933] Node 0 DMA32 per-cpu:
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092935] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092936] CPU 1: hi: 186, btch: 31 usd: 18
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092938] active_anon:932813 inactive_anon:1393 isolated_anon:0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092939] active_file:0 inactive_file:0 isolated_file:0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092940] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092940] free:4048 slab_reclaimable:2409 slab_unreclaimable:2629
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092941] mapped:1357 shmem:1405 pagetables:3318 bounce:0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092943] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092949] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092952] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731252kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9636kB slab_unreclaimable:10516kB kernel_stack:784kB pagetables:13272kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092958] lowmem_reserve[]: 0 0 0 0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092961] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092968] Node 0 DMA32: 946*4kB 238*8kB 8*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7896kB
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092974] 1412 total pagecache pages
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092976] 0 pages in swap cache
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092977] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092978] Free swap = 0kB
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.092979] Total swap = 0kB
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098399] 985072 pages RAM
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098402] 24998 pages reserved
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098403] 3783 pages shared
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098404] 953319 pages non-shared
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098406] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098415] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098418] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098421] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098424] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098427] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098431] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098434] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098437] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098441] [ 790] 101 790 63477 646 0 0 0 rsyslogd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098444] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098447] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098450] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098453] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098456] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098460] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098463] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098466] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098469] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098472] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098475] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098478] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098481] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098484] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098487] [13468] 0 13468 89584 2513 1 0 0 php5-fpm
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098490] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098493] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098497] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098500] [15779] 108 15779 6792 76 1 0 0 pickup
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098503] [18758] 1012 18758 815009 751793 0 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098507] [21011] 1012 21011 69841 6862 0 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098510] [21249] 1012 21249 1099 25 1 0 0 sh
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098513] [21250] 1012 21250 68825 5871 1 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098516] [21256] 1012 21256 1099 26 1 0 0 sh
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098519] [21257] 1012 21257 4658 62 0 0 0 git
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098522] [21258] 1012 21258 68499 5551 1 0 0 php
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098525] [21260] 1012 21260 1099 25 1 0 0 sh
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098528] [21261] 1012 21261 10901 176 0 0 0 ssh
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098531] Out of memory: Kill process 18758 (php) score 784 or sacrifice child
Jul 16 03:56:25 ip-10-61-174-46 kernel: [1473881.098542] Killed process 18758 (php) total-vm:3260036kB, anon-rss:3007168kB, file-rss:4kB
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805687] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805692] php cpuset=/ mems_allowed=0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805695] Pid: 29545, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805697] Call Trace:
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805704] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805708] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805711] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805714] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805720] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805724] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805729] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805732] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805735] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805740] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805743] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805747] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805749] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805752] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805757] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805761] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805766] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805769] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805772] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805776] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805779] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805781] Mem-Info:
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805783] Node 0 DMA per-cpu:
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805785] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805787] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805788] Node 0 DMA32 per-cpu:
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805791] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805793] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805796] active_anon:932319 inactive_anon:1393 isolated_anon:0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805797] active_file:8 inactive_file:17 isolated_file:0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805798] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805799] free:4048 slab_reclaimable:2386 slab_unreclaimable:2682
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805800] mapped:1352 shmem:1405 pagetables:3726 bounce:0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805802] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805811] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805815] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729276kB inactive_anon:5572kB active_file:32kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9544kB slab_unreclaimable:10728kB kernel_stack:888kB pagetables:14904kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:153 all_unreclaimable? yes
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805823] lowmem_reserve[]: 0 0 0 0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805827] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805836] Node 0 DMA32: 1175*4kB 141*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7924kB
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805844] 1430 total pagecache pages
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805846] 0 pages in swap cache
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805847] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805849] Free swap = 0kB
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.805850] Total swap = 0kB
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811551] 985072 pages RAM
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811553] 24998 pages reserved
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811554] 3784 pages shared
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811556] 953316 pages non-shared
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811557] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811565] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811569] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811572] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811575] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811578] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811581] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811585] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811588] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811592] [ 790] 101 790 63477 646 1 0 0 rsyslogd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811595] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811598] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811601] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811605] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811608] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811611] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811614] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811617] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811620] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811623] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811626] [ 654] 1012 654 70351 5771 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811629] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811633] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811636] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811639] [13468] 0 13468 89584 2498 1 0 0 php5-fpm
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811642] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811645] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811648] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811652] [15779] 108 15779 6792 76 1 0 0 pickup
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811655] [21316] 1012 21316 808862 745584 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811658] [28740] 108 28740 6818 72 1 0 0 cleanup
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811661] [28741] 108 28741 6795 71 1 0 0 trivial-rewrite
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811664] [28742] 108 28742 11107 159 0 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811667] [29343] 108 29343 11107 159 0 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811670] [29347] 108 29347 11107 157 1 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811674] [29351] 108 29351 11107 141 0 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811677] [29355] 108 29355 11107 159 1 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811679] [29362] 108 29362 11108 159 0 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811683] [29363] 108 29363 6792 69 1 0 0 scache
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811686] [29370] 108 29370 11108 158 0 0 0 smtp
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811689] [29545] 1012 29545 68580 5605 0 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811692] [29654] 1012 29654 1099 24 0 0 0 sh
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811695] [29655] 1012 29655 1099 25 1 0 0 sh
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811698] [29656] 1012 29656 68825 5870 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811701] [29657] 1012 29657 68825 5873 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811704] [29668] 1012 29668 1099 26 0 0 0 sh
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811706] [29669] 1012 29669 14040 63 1 0 0 git
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811709] [29670] 1012 29670 1099 26 1 0 0 sh
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811712] [29671] 1012 29671 65118 2689 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811715] [29672] 1012 29672 4724 58 1 0 0 git
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811718] [29673] 1012 29673 65118 2688 1 0 0 php
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811721] Out of memory: Kill process 21316 (php) score 778 or sacrifice child
Jul 16 04:05:15 ip-10-61-174-46 kernel: [1474410.811732] Killed process 21316 (php) total-vm:3235448kB, anon-rss:2982332kB, file-rss:4kB
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748499] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748503] php cpuset=/ mems_allowed=0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748506] Pid: 29871, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748507] Call Trace:
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748514] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748516] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748518] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748521] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748525] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748528] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748531] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748534] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748536] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748538] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748541] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748543] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748546] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748549] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748551] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748553] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748556] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748559] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748561] Mem-Info:
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748562] Node 0 DMA per-cpu:
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748564] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748566] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748567] Node 0 DMA32 per-cpu:
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748568] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748570] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748572] active_anon:933006 inactive_anon:1393 isolated_anon:0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748573] active_file:16 inactive_file:0 isolated_file:0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748574] unevictable:0 dirty:4 writeback:0 unstable:0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748574] free:4048 slab_reclaimable:2311 slab_unreclaimable:2636
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748575] mapped:1372 shmem:1405 pagetables:3275 bounce:0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748577] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748583] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748586] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732024kB inactive_anon:5572kB active_file:64kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:16kB writeback:0kB mapped:5488kB shmem:5620kB slab_reclaimable:9244kB slab_unreclaimable:10544kB kernel_stack:736kB pagetables:13100kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:104 all_unreclaimable? yes
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748592] lowmem_reserve[]: 0 0 0 0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748595] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748602] Node 0 DMA32: 953*4kB 251*8kB 5*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7980kB
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748608] 1445 total pagecache pages
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748610] 0 pages in swap cache
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748611] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748612] Free swap = 0kB
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.748613] Total swap = 0kB
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754047] 985072 pages RAM
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754050] 24998 pages reserved
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754051] 3774 pages shared
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754052] 953327 pages non-shared
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754054] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754063] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754066] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754069] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754072] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754076] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754079] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754082] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754086] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754090] [ 790] 101 790 63477 646 1 0 0 rsyslogd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754093] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754096] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754099] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754102] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754105] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754109] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754111] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754115] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754118] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754121] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754124] [ 654] 1012 654 70351 5771 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754127] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754130] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754134] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754137] [13468] 0 13468 89584 2532 1 0 0 php5-fpm
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754140] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754142] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754146] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754149] [15779] 108 15779 6792 76 0 0 0 pickup
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754152] [29363] 108 29363 6792 69 0 0 0 scache
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754155] [29871] 1012 29871 818466 755239 0 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754159] [31047] 1012 31047 68580 5606 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754162] [31176] 1012 31176 1099 24 1 0 0 sh
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754165] [31177] 1012 31177 67882 4970 1 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754168] [31178] 1012 31178 1099 24 1 0 0 sh
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754171] [31179] 1012 31179 67622 4666 0 0 0 php
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754173] Out of memory: Kill process 29871 (php) score 788 or sacrifice child
Jul 16 04:06:48 ip-10-61-174-46 kernel: [1474503.754185] Killed process 29871 (php) total-vm:3273864kB, anon-rss:3020952kB, file-rss:4kB
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320136] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320140] php cpuset=/ mems_allowed=0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320143] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320146] Call Trace:
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320160] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320169] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320173] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320178] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320181] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320184] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320192] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320195] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320198] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320202] [<ffffffff8165d580>] ? down_read+0x10/0x20
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320204] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320208] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320211] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320214] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320217] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320218] Mem-Info:
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320219] Node 0 DMA per-cpu:
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320221] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320223] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320224] Node 0 DMA32 per-cpu:
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320225] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320227] CPU 1: hi: 186, btch: 31 usd: 1
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320229] active_anon:932887 inactive_anon:1393 isolated_anon:0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320230] active_file:0 inactive_file:32 isolated_file:0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320231] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320231] free:4048 slab_reclaimable:2513 slab_unreclaimable:2602
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320232] mapped:1361 shmem:1405 pagetables:3193 bounce:0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320234] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320240] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320243] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731548kB inactive_anon:5572kB active_file:0kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10052kB slab_unreclaimable:10408kB kernel_stack:752kB pagetables:12772kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2671 all_unreclaimable? yes
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320250] lowmem_reserve[]: 0 0 0 0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320252] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320259] Node 0 DMA32: 749*4kB 113*8kB 76*16kB 25*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8028kB
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320266] 1439 total pagecache pages
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320267] 0 pages in swap cache
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320268] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320269] Free swap = 0kB
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.320270] Total swap = 0kB
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325752] 985072 pages RAM
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325755] 24998 pages reserved
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325756] 3791 pages shared
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325758] 953299 pages non-shared
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325760] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325768] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325772] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325775] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325778] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325781] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325785] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325788] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325791] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325795] [ 790] 101 790 63477 646 1 0 0 rsyslogd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325799] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325802] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325805] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325808] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325811] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325814] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325817] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325820] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325824] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325827] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325830] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325833] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325836] [ 658] 1012 658 199421 134839 1 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325839] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325842] [13468] 0 13468 89584 2498 0 0 0 php5-fpm
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325845] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325848] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325851] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325855] [15779] 108 15779 6792 76 0 0 0 pickup
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325858] [29363] 108 29363 6792 69 0 0 0 scache
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325861] [31299] 1012 31299 819362 756137 0 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325864] [31693] 1012 31693 71448 8511 0 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325867] [31829] 1012 31829 1099 25 1 0 0 sh
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325870] [31830] 1012 31830 68695 5717 1 0 0 php
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325873] Out of memory: Kill process 31299 (php) score 789 or sacrifice child
Jul 16 04:07:31 ip-10-61-174-46 kernel: [1474547.325884] Killed process 31299 (php) total-vm:3277448kB, anon-rss:3024548kB, file-rss:0kB
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533678] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533682] php5-fpm cpuset=/ mems_allowed=0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533685] Pid: 13468, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533687] Call Trace:
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533694] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533697] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533699] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533702] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533705] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533709] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533711] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533714] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533718] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533722] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533724] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533727] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533729] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533732] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533734] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533737] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533741] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533745] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533748] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533750] Mem-Info:
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533751] Node 0 DMA per-cpu:
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533753] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533754] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533756] Node 0 DMA32 per-cpu:
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533757] CPU 0: hi: 186, btch: 31 usd: 87
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533759] CPU 1: hi: 186, btch: 31 usd: 105
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533761] active_anon:931841 inactive_anon:1393 isolated_anon:0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533762] active_file:1 inactive_file:7 isolated_file:22
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533763] unevictable:0 dirty:19 writeback:0 unstable:0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533763] free:4014 slab_reclaimable:2871 slab_unreclaimable:2704
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533764] mapped:1352 shmem:1405 pagetables:3547 bounce:0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533766] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533772] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533775] Node 0 DMA32 free:7720kB min:7848kB low:9808kB high:11772kB active_anon:3727364kB inactive_anon:5572kB active_file:4kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:76kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:11484kB slab_unreclaimable:10816kB kernel_stack:840kB pagetables:14188kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:273 all_unreclaimable? yes
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533782] lowmem_reserve[]: 0 0 0 0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533785] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533791] Node 0 DMA32: 1418*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7720kB
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533798] 1468 total pagecache pages
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533799] 0 pages in swap cache
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533800] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533801] Free swap = 0kB
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.533802] Total swap = 0kB
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539346] 985072 pages RAM
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539348] 24998 pages reserved
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539349] 3782 pages shared
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539350] 953183 pages non-shared
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539352] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539361] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539364] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539367] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539370] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539373] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539377] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539380] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539383] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539387] [ 790] 101 790 63477 646 0 0 0 rsyslogd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539390] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539393] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539397] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539400] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539403] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539406] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539409] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539412] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539415] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539419] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539422] [ 654] 1012 654 70351 5771 0 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539425] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539428] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539431] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539434] [13468] 0 13468 89584 2531 0 0 0 php5-fpm
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539437] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539440] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539443] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539446] [15779] 108 15779 6792 76 0 0 0 pickup
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539450] [31989] 1012 31989 803662 740333 0 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539453] [ 6906] 1012 6906 68580 5604 0 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539456] [ 7117] 1012 7117 1099 25 1 0 0 sh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539459] [ 7118] 1012 7118 68825 5874 1 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539462] [ 7119] 1012 7119 1099 24 1 0 0 sh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539465] [ 7120] 1012 7120 68825 5871 1 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539468] [ 7130] 1012 7130 1099 26 1 0 0 sh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539472] [ 7132] 1012 7132 14040 63 1 0 0 git
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539474] [ 7133] 1012 7133 68499 5551 1 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539477] [ 7134] 1012 7134 1099 26 1 0 0 sh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539480] [ 7135] 1012 7135 4669 68 1 0 0 git
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539483] [ 7136] 1012 7136 68499 5552 1 0 0 php
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539486] [ 7139] 1012 7139 1099 24 0 0 0 sh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539489] [ 7140] 1012 7140 1099 24 0 0 0 sh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539492] [ 7141] 1012 7141 10901 176 0 0 0 ssh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539495] [ 7142] 1012 7142 10901 175 0 0 0 ssh
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539498] Out of memory: Kill process 31989 (php) score 772 or sacrifice child
Jul 16 04:15:07 ip-10-61-174-46 kernel: [1475002.539509] Killed process 31989 (php) total-vm:3214648kB, anon-rss:2961332kB, file-rss:0kB
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736113] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736117] php cpuset=/ mems_allowed=0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736119] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736121] Call Trace:
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736127] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736130] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736132] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736135] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736138] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736142] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736144] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736147] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736151] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736154] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736157] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736159] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736162] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736164] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736167] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736169] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736173] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736176] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736178] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736181] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736183] Mem-Info:
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736184] Node 0 DMA per-cpu:
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736186] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736187] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736188] Node 0 DMA32 per-cpu:
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736190] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736191] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736193] active_anon:933181 inactive_anon:1393 isolated_anon:0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736194] active_file:0 inactive_file:0 isolated_file:0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736195] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736195] free:4048 slab_reclaimable:2399 slab_unreclaimable:2604
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736196] mapped:1334 shmem:1405 pagetables:3072 bounce:0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736198] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736204] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736207] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732724kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5336kB shmem:5620kB slab_reclaimable:9596kB slab_unreclaimable:10416kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736213] lowmem_reserve[]: 0 0 0 0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736216] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736223] Node 0 DMA32: 654*4kB 159*8kB 119*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7904kB
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736229] 1433 total pagecache pages
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736231] 0 pages in swap cache
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736232] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736233] Free swap = 0kB
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.736234] Total swap = 0kB
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741725] 985072 pages RAM
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741727] 24998 pages reserved
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741729] 3778 pages shared
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741730] 953331 pages non-shared
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741732] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741741] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741744] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741747] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741750] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741753] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741757] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741760] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741763] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741767] [ 790] 101 790 63477 646 0 0 0 rsyslogd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741770] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741773] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741777] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741780] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741783] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741786] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741789] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741792] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741796] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741799] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741802] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741805] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741808] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741811] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741814] [13468] 0 13468 89584 2539 0 0 0 php5-fpm
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741817] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741820] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741823] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741827] [15779] 108 15779 6792 76 1 0 0 pickup
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741830] [ 7209] 1012 7209 828386 765161 0 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741833] [ 8363] 1012 8363 68580 5604 1 0 0 php
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741836] Out of memory: Kill process 7209 (php) score 798 or sacrifice child
Jul 16 04:16:32 ip-10-61-174-46 kernel: [1475087.741847] Killed process 7209 (php) total-vm:3313544kB, anon-rss:3060640kB, file-rss:4kB
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552144] php cpuset=/ mems_allowed=0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552147] Pid: 8650, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552149] Call Trace:
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552170] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552173] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552177] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552182] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552185] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552187] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552191] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552193] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552195] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552202] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552204] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552207] [<ffffffff8165c2f1>] ? __schedule+0x6b1/0x6f0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552210] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552211] Mem-Info:
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552213] Node 0 DMA per-cpu:
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552215] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552216] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552217] Node 0 DMA32 per-cpu:
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552219] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552220] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552222] active_anon:932318 inactive_anon:1393 isolated_anon:0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552223] active_file:0 inactive_file:4 isolated_file:0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552224] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552224] free:4048 slab_reclaimable:2770 slab_unreclaimable:2657
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552225] mapped:1376 shmem:1405 pagetables:3389 bounce:0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552227] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552233] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552236] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729272kB inactive_anon:5572kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5504kB shmem:5620kB slab_reclaimable:11080kB slab_unreclaimable:10628kB kernel_stack:776kB pagetables:13556kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2040 all_unreclaimable? yes
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552242] lowmem_reserve[]: 0 0 0 0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552251] Node 0 DMA32: 1072*4kB 197*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7944kB
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552258] 1424 total pagecache pages
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552259] 0 pages in swap cache
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552261] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552262] Free swap = 0kB
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.552262] Total swap = 0kB
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557626] 985072 pages RAM
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557629] 24998 pages reserved
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557630] 3791 pages shared
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557631] 953312 pages non-shared
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557633] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557642] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557646] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557649] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557652] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557655] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557658] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557662] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557665] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557669] [ 790] 101 790 63477 646 1 0 0 rsyslogd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557673] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557676] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557679] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557682] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557685] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557689] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557691] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557695] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557698] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557701] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557704] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557707] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557710] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557713] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557716] [13468] 0 13468 89584 2524 1 0 0 php5-fpm
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557719] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557722] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557725] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557729] [15779] 108 15779 6792 76 1 0 0 pickup
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557733] [ 8650] 1012 8650 820619 757411 0 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557736] [ 9911] 108 9911 6818 74 1 0 0 cleanup
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557739] [ 9912] 108 9912 6795 71 1 0 0 trivial-rewrite
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557742] [ 9913] 108 9913 11107 159 0 0 0 smtp
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557745] [10014] 108 10014 11107 159 0 0 0 smtp
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557748] [10018] 108 10018 11107 159 0 0 0 smtp
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557751] [10022] 108 10022 6792 69 1 0 0 scache
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557754] [10857] 1012 10857 73461 10552 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557757] [11076] 1012 11076 1099 24 1 0 0 sh
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557760] [11077] 1012 11077 1099 24 0 0 0 sh
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557763] [11078] 1012 11078 1099 24 1 0 0 sh
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557766] [11079] 1012 11079 29626 374 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557769] [11080] 1012 11080 31456 391 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557772] [11081] 1012 11081 31456 383 1 0 0 php
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557775] Out of memory: Kill process 8650 (php) score 790 or sacrifice child
Jul 16 04:19:12 ip-10-61-174-46 kernel: [1475247.557786] Killed process 8650 (php) total-vm:3282476kB, anon-rss:3029644kB, file-rss:0kB
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952103] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952107] php cpuset=/ mems_allowed=0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952110] Pid: 11183, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952112] Call Trace:
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952119] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952121] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952123] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952126] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952131] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952134] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952138] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952140] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952143] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952145] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952148] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952151] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952153] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952156] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952158] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952162] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952164] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952166] [<ffffffff811425f9>] ? find_vma+0x49/0x70
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952169] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952171] Mem-Info:
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952172] Node 0 DMA per-cpu:
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952174] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952175] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952176] Node 0 DMA32 per-cpu:
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952178] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952179] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952182] active_anon:932628 inactive_anon:1393 isolated_anon:0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952183] active_file:14 inactive_file:21 isolated_file:0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952183] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952184] free:4048 slab_reclaimable:2624 slab_unreclaimable:2683
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952185] mapped:1353 shmem:1405 pagetables:3257 bounce:0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952186] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952193] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952196] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730512kB inactive_anon:5572kB active_file:56kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:10496kB slab_unreclaimable:10732kB kernel_stack:792kB pagetables:13028kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14265 all_unreclaimable? yes
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952202] lowmem_reserve[]: 0 0 0 0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952205] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952211] Node 0 DMA32: 943*4kB 266*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8044kB
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952218] 1433 total pagecache pages
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952219] 0 pages in swap cache
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952221] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952222] Free swap = 0kB
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.952223] Total swap = 0kB
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957833] 985072 pages RAM
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957835] 24998 pages reserved
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957836] 3792 pages shared
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957837] 953291 pages non-shared
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957839] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957848] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957852] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957855] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957858] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957861] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957865] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957868] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957871] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957875] [ 790] 101 790 63477 646 0 0 0 rsyslogd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957878] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957882] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957885] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957888] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957891] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957894] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957897] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957901] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957904] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957907] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957910] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957913] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957916] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957919] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957922] [13468] 0 13468 89584 2509 1 0 0 php5-fpm
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957925] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957928] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957931] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957935] [15779] 108 15779 6792 76 0 0 0 pickup
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957938] [11183] 1012 11183 826751 763569 0 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957941] [12677] 108 12677 6818 74 0 0 0 cleanup
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957945] [12678] 108 12678 6795 70 0 0 0 trivial-rewrite
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957948] [12679] 108 12679 11107 158 0 0 0 smtp
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957951] [12683] 108 12683 11107 158 0 0 0 smtp
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957954] [12687] 108 12687 11107 157 1 0 0 smtp
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957957] [12696] 108 12696 11107 157 0 0 0 smtp
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957960] [12713] 108 12713 11107 158 0 0 0 smtp
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957963] [12726] 108 12726 6792 69 0 0 0 scache
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957966] [13729] 1012 13729 68580 5605 1 0 0 php
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957968] Out of memory: Kill process 11183 (php) score 796 or sacrifice child
Jul 16 04:22:04 ip-10-61-174-46 kernel: [1475419.957980] Killed process 11183 (php) total-vm:3307004kB, anon-rss:3054276kB, file-rss:0kB
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276153] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276157] php cpuset=/ mems_allowed=0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276160] Pid: 14039, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276162] Call Trace:
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276168] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276171] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276173] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276177] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276183] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276187] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276191] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276193] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276196] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276199] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276204] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276207] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276210] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276214] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276217] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276221] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276224] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276226] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276230] [<ffffffff8165c2f1>] ? __schedule+0x6b1/0x6f0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276234] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276236] Mem-Info:
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276238] Node 0 DMA per-cpu:
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276240] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276242] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276244] Node 0 DMA32 per-cpu:
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276246] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276248] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276251] active_anon:932617 inactive_anon:1393 isolated_anon:0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276252] active_file:0 inactive_file:31 isolated_file:0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276253] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276254] free:4048 slab_reclaimable:2314 slab_unreclaimable:2640
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276255] mapped:1383 shmem:1405 pagetables:3472 bounce:0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276257] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276266] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276270] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730468kB inactive_anon:5572kB active_file:0kB inactive_file:124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5532kB shmem:5620kB slab_reclaimable:9256kB slab_unreclaimable:10560kB kernel_stack:832kB pagetables:13888kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4828 all_unreclaimable? yes
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276278] lowmem_reserve[]: 0 0 0 0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276281] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276291] Node 0 DMA32: 739*4kB 194*8kB 43*16kB 18*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8012kB
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276300] 1463 total pagecache pages
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276302] 0 pages in swap cache
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276303] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276305] Free swap = 0kB
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.276306] Total swap = 0kB
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281791] 985072 pages RAM
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281793] 24998 pages reserved
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281795] 3797 pages shared
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281796] 953258 pages non-shared
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281798] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281806] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281810] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281813] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281816] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281819] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281822] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281826] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281829] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281833] [ 790] 101 790 63477 646 0 0 0 rsyslogd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281836] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281840] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281843] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281846] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281849] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281852] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281855] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281858] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281861] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281864] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281867] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281870] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281874] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281877] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281880] [13468] 0 13468 89584 2534 0 0 0 php5-fpm
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281883] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281886] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281889] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281893] [15779] 108 15779 6792 76 0 0 0 pickup
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281896] [12726] 108 12726 6792 69 0 0 0 scache
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281899] [14039] 1012 14039 809889 746639 1 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281902] [14531] 1012 14531 68580 5606 0 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281905] [14670] 1012 14670 1099 25 0 0 0 sh
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281908] [14671] 1012 14671 68825 5869 0 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281911] [14672] 1012 14672 1099 25 0 0 0 sh
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281914] [14673] 1012 14673 68825 5889 1 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281917] [14685] 1012 14685 1099 27 0 0 0 sh
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281920] [14687] 1012 14687 4767 93 1 0 0 git
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281923] [14689] 1012 14689 68499 5552 0 0 0 php
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281926] [14694] 1012 14694 1099 25 0 0 0 sh
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281929] [14695] 1012 14695 10901 165 0 0 0 ssh
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281932] [14700] 1012 14700 40074 113 0 0 0 git
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281935] [14701] 1012 14701 1099 25 1 0 0 sh
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281938] [14702] 1012 14702 4647 53 0 0 0 git
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281940] Out of memory: Kill process 14039 (php) score 779 or sacrifice child
Jul 16 04:22:58 ip-10-61-174-46 kernel: [1475474.281951] Killed process 14039 (php) total-vm:3239556kB, anon-rss:2986556kB, file-rss:0kB
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068166] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068170] php cpuset=/ mems_allowed=0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068173] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068175] Call Trace:
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068182] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068184] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068186] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068189] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068192] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068196] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068198] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068201] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068204] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068207] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068210] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068212] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068215] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068218] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068222] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068224] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068227] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068230] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068231] Mem-Info:
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068233] Node 0 DMA per-cpu:
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068234] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068236] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068237] Node 0 DMA32 per-cpu:
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068239] CPU 0: hi: 186, btch: 31 usd: 44
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068240] CPU 1: hi: 186, btch: 31 usd: 62
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068242] active_anon:932832 inactive_anon:1393 isolated_anon:0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068243] active_file:12 inactive_file:12 isolated_file:0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068244] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068244] free:4040 slab_reclaimable:2644 slab_unreclaimable:2657
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068245] mapped:1396 shmem:1405 pagetables:3088 bounce:0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068247] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068253] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068256] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3731328kB inactive_anon:5572kB active_file:48kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5584kB shmem:5620kB slab_reclaimable:10576kB slab_unreclaimable:10628kB kernel_stack:736kB pagetables:12352kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:148 all_unreclaimable? yes
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068263] lowmem_reserve[]: 0 0 0 0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068265] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068272] Node 0 DMA32: 1110*4kB 167*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068279] 1435 total pagecache pages
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068280] 0 pages in swap cache
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068281] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068282] Free swap = 0kB
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.068283] Total swap = 0kB
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073746] 985072 pages RAM
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073749] 24998 pages reserved
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073750] 3788 pages shared
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073751] 953250 pages non-shared
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073753] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073762] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073765] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073768] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073772] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073775] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073778] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073782] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073785] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073789] [ 790] 101 790 63477 647 0 0 0 rsyslogd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073792] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073795] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073799] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073802] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073805] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073808] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073811] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073814] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073817] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073820] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073824] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073827] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073830] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073833] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073836] [13468] 0 13468 89584 2519 1 0 0 php5-fpm
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073839] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073842] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073845] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073849] [14927] 1012 14927 827986 764732 1 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073852] [16186] 108 16186 6792 69 1 0 0 pickup
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073855] [16194] 108 16194 6792 70 1 0 0 scache
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073858] [17676] 1012 17676 68580 5603 0 0 0 php
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073861] Out of memory: Kill process 14927 (php) score 798 or sacrifice child
Jul 16 04:26:03 ip-10-61-174-46 kernel: [1475659.073872] Killed process 14927 (php) total-vm:3311944kB, anon-rss:3058928kB, file-rss:0kB
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161727] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161730] php cpuset=/ mems_allowed=0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161733] Pid: 17991, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161734] Call Trace:
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161741] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161744] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161746] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161749] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161752] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161755] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161757] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161761] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161765] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161767] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161771] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161773] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161775] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161778] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161780] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161783] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161785] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161787] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161790] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161793] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161795] Mem-Info:
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161796] Node 0 DMA per-cpu:
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161797] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161799] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161800] Node 0 DMA32 per-cpu:
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161801] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161803] CPU 1: hi: 186, btch: 31 usd: 19
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161805] active_anon:933251 inactive_anon:1393 isolated_anon:0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161806] active_file:12 inactive_file:0 isolated_file:0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161807] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161807] free:4048 slab_reclaimable:2283 slab_unreclaimable:2604
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161808] mapped:1366 shmem:1405 pagetables:3094 bounce:0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161810] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161816] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161819] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3733004kB inactive_anon:5572kB active_file:48kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5464kB shmem:5620kB slab_reclaimable:9132kB slab_unreclaimable:10416kB kernel_stack:736kB pagetables:12376kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:100 all_unreclaimable? yes
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161826] lowmem_reserve[]: 0 0 0 0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161828] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161835] Node 0 DMA32: 572*4kB 223*8kB 100*16kB 7*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7944kB
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161841] 1450 total pagecache pages
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161843] 0 pages in swap cache
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161844] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161845] Free swap = 0kB
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.161846] Total swap = 0kB
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167293] 985072 pages RAM
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167295] 24998 pages reserved
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167296] 3790 pages shared
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167297] 953306 pages non-shared
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167299] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167308] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167311] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167315] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167318] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167321] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167324] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167327] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167331] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167335] [ 790] 101 790 63477 647 0 0 0 rsyslogd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167338] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167341] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167344] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167347] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167350] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167353] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167357] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167360] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167363] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167366] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167369] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167372] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167375] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167378] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167382] [13468] 0 13468 89584 2521 0 0 0 php5-fpm
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167385] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167388] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167391] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167395] [16186] 108 16186 6792 69 0 0 0 pickup
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167398] [16194] 108 16194 6792 70 1 0 0 scache
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167400] [17991] 1012 17991 828386 765139 0 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167403] [18013] 1012 18013 68580 5604 0 0 0 php
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167406] Out of memory: Kill process 17991 (php) score 798 or sacrifice child
Jul 16 04:26:36 ip-10-61-174-46 kernel: [1475692.167417] Killed process 17991 (php) total-vm:3313544kB, anon-rss:3060556kB, file-rss:0kB
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308103] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308106] php cpuset=/ mems_allowed=0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308109] Pid: 18013, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308111] Call Trace:
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308118] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308120] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308122] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308125] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308128] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308131] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308134] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308137] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308140] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308142] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308145] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308147] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308150] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308152] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308154] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308157] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308160] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308162] Mem-Info:
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308163] Node 0 DMA per-cpu:
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308165] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308166] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308167] Node 0 DMA32 per-cpu:
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308169] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308170] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308173] active_anon:932759 inactive_anon:1393 isolated_anon:0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308173] active_file:0 inactive_file:14 isolated_file:0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308174] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308175] free:4048 slab_reclaimable:2644 slab_unreclaimable:2651
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308175] mapped:1374 shmem:1405 pagetables:3223 bounce:0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308177] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308183] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308186] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731036kB inactive_anon:5572kB active_file:0kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5620kB slab_reclaimable:10576kB slab_unreclaimable:10604kB kernel_stack:776kB pagetables:12892kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3388 all_unreclaimable? yes
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308193] lowmem_reserve[]: 0 0 0 0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308195] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308202] Node 0 DMA32: 1002*4kB 215*8kB 5*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308208] 1452 total pagecache pages
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308210] 0 pages in swap cache
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308211] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308212] Free swap = 0kB
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.308213] Total swap = 0kB
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313705] 985072 pages RAM
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313707] 24998 pages reserved
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313708] 3804 pages shared
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313709] 953328 pages non-shared
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313711] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313720] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313724] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313727] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313730] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313733] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313736] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313740] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313743] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313747] [ 790] 101 790 63477 647 0 0 0 rsyslogd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313750] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313753] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313756] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313759] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313763] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313766] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313769] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313772] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313775] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313778] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313781] [ 654] 1012 654 70351 5770 1 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313784] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313787] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313790] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313794] [13468] 0 13468 89584 2506 1 0 0 php5-fpm
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313797] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313800] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313803] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313806] [16186] 108 16186 6792 75 0 0 0 pickup
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313809] [18013] 1012 18013 827212 763944 0 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313812] [19541] 108 19541 6818 73 0 0 0 cleanup
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313816] [19542] 108 19542 6795 71 0 0 0 trivial-rewrite
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313819] [19543] 108 19543 11107 159 0 0 0 smtp
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313822] [19547] 108 19547 11107 159 0 0 0 smtp
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313825] [19551] 108 19551 11107 158 0 0 0 smtp
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313828] [19555] 108 19555 11107 159 1 0 0 smtp
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313831] [19879] 1012 19879 68580 5603 1 0 0 php
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313834] Out of memory: Kill process 18013 (php) score 797 or sacrifice child
Jul 16 04:28:27 ip-10-61-174-46 kernel: [1475803.313846] Killed process 18013 (php) total-vm:3308848kB, anon-rss:3055676kB, file-rss:100kB
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940171] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940175] php cpuset=/ mems_allowed=0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940178] Pid: 22051, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940181] Call Trace:
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940188] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940192] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940195] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940199] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940205] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940208] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940213] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940216] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940220] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940223] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940227] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940231] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940234] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940238] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940241] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940245] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940249] [<ffffffff81196e8e>] ? vfsmount_lock_local_unlock+0x1e/0x30
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940253] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940256] [<ffffffff81198d7f>] ? mntput+0x1f/0x30
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940260] [<ffffffff8117aca3>] ? __fput+0x143/0x200
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940263] [<ffffffff8117ad85>] ? fput+0x25/0x30
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940267] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940269] Mem-Info:
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940271] Node 0 DMA per-cpu:
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940273] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940275] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940277] Node 0 DMA32 per-cpu:
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940279] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940281] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940284] active_anon:932406 inactive_anon:1393 isolated_anon:0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940285] active_file:0 inactive_file:12 isolated_file:0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940286] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940287] free:4048 slab_reclaimable:2521 slab_unreclaimable:2673
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940288] mapped:1355 shmem:1405 pagetables:3498 bounce:0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940290] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940299] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940302] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729624kB inactive_anon:5572kB active_file:0kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10084kB slab_unreclaimable:10692kB kernel_stack:808kB pagetables:13992kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:196 all_unreclaimable? yes
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940323] lowmem_reserve[]: 0 0 0 0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940326] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940334] Node 0 DMA32: 827*4kB 149*8kB 37*16kB 10*32kB 4*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7972kB
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940340] 1455 total pagecache pages
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940341] 0 pages in swap cache
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940343] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940344] Free swap = 0kB
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.940345] Total swap = 0kB
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945946] 985072 pages RAM
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945948] 24998 pages reserved
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945950] 3792 pages shared
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945951] 953306 pages non-shared
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945953] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945962] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945965] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945968] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945972] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945975] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945978] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945981] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945985] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945989] [ 790] 101 790 63477 647 1 0 0 rsyslogd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945992] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945995] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.945999] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946002] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946005] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946008] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946011] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946014] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946017] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946020] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946024] [ 654] 1012 654 70351 5771 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946027] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946030] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946033] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946036] [13468] 0 13468 89584 2535 1 0 0 php5-fpm
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946039] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946042] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946045] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946049] [16186] 108 16186 6792 75 0 0 0 pickup
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946052] [20120] 1012 20120 805729 742487 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946055] [21871] 1012 21871 68580 5605 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946058] [22012] 1012 22012 1099 24 1 0 0 sh
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946061] [22013] 1012 22013 68911 5880 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946064] [22014] 1012 22014 1099 24 1 0 0 sh
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946067] [22015] 1012 22015 68912 5880 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946070] [22044] 1012 22044 1099 26 1 0 0 sh
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946073] [22046] 1012 22046 4675 59 1 0 0 git
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946076] [22048] 1012 22048 67880 4944 1 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946079] [22049] 1012 22049 1099 26 0 0 0 sh
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946082] [22050] 1012 22050 4682 58 1 0 0 git
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946085] [22051] 1012 22051 67880 4943 0 0 0 php
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946088] Out of memory: Kill process 20120 (php) score 774 or sacrifice child
Jul 16 04:30:41 ip-10-61-174-46 kernel: [1475936.946099] Killed process 20120 (php) total-vm:3222916kB, anon-rss:2969948kB, file-rss:0kB
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904130] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904135] php cpuset=/ mems_allowed=0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904138] Pid: 23783, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904140] Call Trace:
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904152] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904157] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904162] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904165] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904169] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904171] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904174] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904176] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904179] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904181] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904184] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904186] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904191] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904194] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904197] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904200] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904202] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904205] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904207] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904209] Mem-Info:
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904210] Node 0 DMA per-cpu:
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904212] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904214] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904215] Node 0 DMA32 per-cpu:
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904216] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904218] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904220] active_anon:932822 inactive_anon:1393 isolated_anon:0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904221] active_file:12 inactive_file:26 isolated_file:0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904221] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904222] free:4048 slab_reclaimable:2590 slab_unreclaimable:2625
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904223] mapped:1361 shmem:1405 pagetables:3187 bounce:0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904224] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904231] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904234] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731288kB inactive_anon:5572kB active_file:48kB inactive_file:104kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10360kB slab_unreclaimable:10500kB kernel_stack:768kB pagetables:12748kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:700 all_unreclaimable? yes
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904240] lowmem_reserve[]: 0 0 0 0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904243] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904249] Node 0 DMA32: 990*4kB 239*8kB 5*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8032kB
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904256] 1422 total pagecache pages
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904257] 0 pages in swap cache
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904258] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904259] Free swap = 0kB
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.904260] Total swap = 0kB
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909726] 985072 pages RAM
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909729] 24998 pages reserved
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909730] 3772 pages shared
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909731] 953316 pages non-shared
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909733] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909741] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909745] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909748] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909751] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909754] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909758] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909761] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909764] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909768] [ 790] 101 790 63477 647 0 0 0 rsyslogd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909771] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909775] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909778] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909781] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909784] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909787] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909790] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909793] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909796] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909799] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909802] [ 654] 1012 654 70351 5779 0 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909805] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909808] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909811] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909814] [13468] 0 13468 89584 2517 0 0 0 php5-fpm
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909817] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909820] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909824] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909827] [16186] 108 16186 6792 75 1 0 0 pickup
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909830] [22130] 1012 22130 827353 764144 1 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909833] [23306] 108 23306 6818 73 1 0 0 cleanup
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909837] [23307] 108 23307 6795 70 1 0 0 trivial-rewrite
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909840] [23308] 108 23308 11107 159 0 0 0 smtp
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909843] [23312] 108 23312 11107 143 0 0 0 smtp
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909846] [23316] 108 23316 11107 158 1 0 0 smtp
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909849] [23783] 1012 23783 68580 5605 0 0 0 php
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909852] Out of memory: Kill process 22130 (php) score 797 or sacrifice child
Jul 16 04:32:56 ip-10-61-174-46 kernel: [1476071.909863] Killed process 22130 (php) total-vm:3309412kB, anon-rss:3056576kB, file-rss:0kB
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900110] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900114] php cpuset=/ mems_allowed=0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900116] Pid: 26189, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900118] Call Trace:
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900125] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900127] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900130] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900132] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900136] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900139] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900141] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900144] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900147] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900150] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900152] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900155] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900157] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900161] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900164] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900168] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900170] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900173] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900175] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900178] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900180] Mem-Info:
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900181] Node 0 DMA per-cpu:
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900183] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900184] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900185] Node 0 DMA32 per-cpu:
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900187] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900188] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900191] active_anon:933161 inactive_anon:1393 isolated_anon:0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900192] active_file:0 inactive_file:0 isolated_file:0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900192] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900193] free:4047 slab_reclaimable:2396 slab_unreclaimable:2607
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900193] mapped:1373 shmem:1405 pagetables:3076 bounce:0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900195] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900202] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900205] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3732644kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5492kB shmem:5620kB slab_reclaimable:9584kB slab_unreclaimable:10428kB kernel_stack:728kB pagetables:12304kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900211] lowmem_reserve[]: 0 0 0 0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900214] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900220] Node 0 DMA32: 925*4kB 257*8kB 9*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900227] 1464 total pagecache pages
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900228] 0 pages in swap cache
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900230] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900231] Free swap = 0kB
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.900232] Total swap = 0kB
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905687] 985072 pages RAM
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905690] 24998 pages reserved
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905691] 3786 pages shared
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905692] 953311 pages non-shared
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905694] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905703] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905706] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905709] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905712] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905715] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905719] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905722] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905725] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905730] [ 790] 101 790 63477 647 1 0 0 rsyslogd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905733] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905736] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905739] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905742] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905745] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905748] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905752] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905755] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905758] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905761] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905764] [ 654] 1012 654 70351 5779 0 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905767] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905770] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905773] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905776] [13468] 0 13468 89584 2529 0 0 0 php5-fpm
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905779] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905782] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905786] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905790] [16186] 108 16186 6792 75 1 0 0 pickup
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905793] [24113] 1012 24113 828378 765124 0 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905796] [26189] 1012 26189 68580 5603 1 0 0 php
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905798] Out of memory: Kill process 24113 (php) score 798 or sacrifice child
Jul 16 04:35:32 ip-10-61-174-46 kernel: [1476227.905810] Killed process 24113 (php) total-vm:3313512kB, anon-rss:3060496kB, file-rss:0kB
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808547] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808551] php cpuset=/ mems_allowed=0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808553] Pid: 26419, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808555] Call Trace:
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808562] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808564] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808567] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808570] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808573] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808576] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808578] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808581] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808584] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808587] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808589] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808592] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808595] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808597] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808601] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808603] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808605] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808608] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808609] Mem-Info:
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808610] Node 0 DMA per-cpu:
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808612] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808614] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808615] Node 0 DMA32 per-cpu:
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808617] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808618] CPU 1: hi: 186, btch: 31 usd: 55
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808620] active_anon:932442 inactive_anon:1393 isolated_anon:0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808621] active_file:0 inactive_file:0 isolated_file:0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808622] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808622] free:4028 slab_reclaimable:2455 slab_unreclaimable:2677
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808623] mapped:1357 shmem:1405 pagetables:3527 bounce:0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808625] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808631] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808634] Node 0 DMA32 free:7776kB min:7848kB low:9808kB high:11772kB active_anon:3729768kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9820kB slab_unreclaimable:10708kB kernel_stack:840kB pagetables:14108kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:58 all_unreclaimable? yes
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808640] lowmem_reserve[]: 0 0 0 0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808643] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808650] Node 0 DMA32: 1252*4kB 90*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7776kB
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808656] 1442 total pagecache pages
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808657] 0 pages in swap cache
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808659] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808660] Free swap = 0kB
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.808661] Total swap = 0kB
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814177] 985072 pages RAM
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814179] 24998 pages reserved
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814181] 3786 pages shared
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814182] 953270 pages non-shared
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814184] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814192] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814196] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814199] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814202] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814205] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814209] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814212] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814215] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814219] [ 790] 101 790 63477 647 1 0 0 rsyslogd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814223] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814226] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814229] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814232] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814235] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814239] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814242] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814245] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814248] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814251] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814254] [ 654] 1012 654 70351 5779 0 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814257] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814260] [ 658] 1012 658 199421 134839 1 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814263] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814266] [13468] 0 13468 89584 2514 0 0 0 php5-fpm
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814270] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814272] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814276] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814279] [16186] 108 16186 6792 75 0 0 0 pickup
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814282] [26419] 1012 26419 814734 751461 0 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814285] [ 1725] 108 1725 6818 73 1 0 0 cleanup
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814288] [ 1726] 108 1726 6795 70 0 0 0 trivial-rewrite
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814291] [ 1727] 108 1727 11107 156 0 0 0 smtp
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814294] [ 1731] 108 1731 11107 160 1 0 0 smtp
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814297] [ 1735] 108 1735 11107 159 0 0 0 smtp
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814300] [ 1739] 108 1739 11107 159 0 0 0 smtp
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814303] [ 4293] 108 4293 11107 158 0 0 0 smtp
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814306] [ 4300] 108 4300 11108 159 0 0 0 smtp
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814309] [ 4301] 108 4301 6792 70 1 0 0 scache
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814312] [ 5266] 1012 5266 68580 5604 1 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814315] [ 5437] 1012 5437 1099 25 0 0 0 sh
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814318] [ 5438] 1012 5438 68825 5870 0 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814321] [ 5444] 1012 5444 1099 26 0 0 0 sh
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814325] [ 5445] 1012 5445 4738 86 0 0 0 git
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814327] [ 5446] 1012 5446 68499 5551 0 0 0 php
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814331] [ 5448] 1012 5448 1099 25 0 0 0 sh
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814333] [ 5449] 1012 5449 10901 165 0 0 0 ssh
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814336] Out of memory: Kill process 26419 (php) score 784 or sacrifice child
Jul 16 04:47:48 ip-10-61-174-46 kernel: [1476963.814348] Killed process 26419 (php) total-vm:3258936kB, anon-rss:3005844kB, file-rss:0kB
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115968] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115973] ntpd cpuset=/ mems_allowed=0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115975] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115977] Call Trace:
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115984] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115987] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115989] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115992] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115995] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.115998] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116000] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116003] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116006] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116009] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116012] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116014] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116017] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116021] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116024] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116026] Mem-Info:
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116027] Node 0 DMA per-cpu:
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116029] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116030] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116031] Node 0 DMA32 per-cpu:
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116033] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116034] CPU 1: hi: 186, btch: 31 usd: 83
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116037] active_anon:932845 inactive_anon:1393 isolated_anon:0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116037] active_file:33 inactive_file:9 isolated_file:21
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116038] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116039] free:4042 slab_reclaimable:2366 slab_unreclaimable:2610
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116039] mapped:1362 shmem:1405 pagetables:3317 bounce:0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116041] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116048] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116063] Node 0 DMA32 free:7832kB min:7848kB low:9808kB high:11772kB active_anon:3731380kB inactive_anon:5572kB active_file:132kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):84kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5620kB slab_reclaimable:9464kB slab_unreclaimable:10440kB kernel_stack:768kB pagetables:13268kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:344 all_unreclaimable? yes
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116072] lowmem_reserve[]: 0 0 0 0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116076] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116086] Node 0 DMA32: 974*4kB 237*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7840kB
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116096] 1448 total pagecache pages
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116098] 0 pages in swap cache
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116100] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116101] Free swap = 0kB
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.116102] Total swap = 0kB
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121617] 985072 pages RAM
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121619] 24998 pages reserved
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121620] 3842 pages shared
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121621] 953181 pages non-shared
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121623] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121630] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121633] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121635] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121637] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121639] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121642] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121644] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121646] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121650] [ 790] 101 790 63477 647 1 0 0 rsyslogd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121652] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121654] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121657] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121659] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121661] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121663] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121665] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121668] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121670] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121672] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121674] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121677] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121679] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121681] [ 678] 1012 678 70152 5567 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121683] [13468] 0 13468 89584 2499 1 0 0 php5-fpm
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121685] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121687] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121689] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121692] [16186] 108 16186 6792 75 0 0 0 pickup
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121695] [ 4301] 108 4301 6792 70 1 0 0 scache
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121697] [ 5530] 1012 5530 661208 597948 0 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121699] [ 6319] 1012 6319 224776 160747 0 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121701] [ 6384] 1012 6384 68580 5599 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121703] [ 6386] 1012 6386 1099 24 0 0 0 sh
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121705] [ 6387] 1012 6387 68825 5872 1 0 0 php
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121707] [ 6393] 1012 6393 1099 25 0 0 0 sh
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121709] [ 6394] 1012 6394 4682 61 0 0 0 git
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121711] [ 6395] 1012 6395 4682 61 1 0 0 git
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121714] Out of memory: Kill process 5530 (php) score 624 or sacrifice child
Jul 16 04:48:55 ip-10-61-174-46 kernel: [1477031.121723] Killed process 5530 (php) total-vm:2644832kB, anon-rss:2391788kB, file-rss:4kB
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873522] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873526] php cpuset=/ mems_allowed=0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873529] Pid: 6479, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873530] Call Trace:
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873537] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873539] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873542] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873545] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873548] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873552] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873555] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873558] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873561] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873564] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873566] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873569] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873571] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873574] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873578] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873580] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873583] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873586] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873588] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873591] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873594] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873595] Mem-Info:
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873596] Node 0 DMA per-cpu:
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873598] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873600] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873601] Node 0 DMA32 per-cpu:
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873602] CPU 0: hi: 186, btch: 31 usd: 64
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873604] CPU 1: hi: 186, btch: 31 usd: 31
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873606] active_anon:932743 inactive_anon:1393 isolated_anon:0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873607] active_file:15 inactive_file:22 isolated_file:10
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873608] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873608] free:4045 slab_reclaimable:2328 slab_unreclaimable:2607
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873609] mapped:1367 shmem:1405 pagetables:3418 bounce:0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873611] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873617] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873620] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3730972kB inactive_anon:5572kB active_file:60kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):40kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5620kB slab_reclaimable:9312kB slab_unreclaimable:10428kB kernel_stack:800kB pagetables:13672kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:205 all_unreclaimable? yes
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873627] lowmem_reserve[]: 0 0 0 0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873629] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873636] Node 0 DMA32: 445*4kB 339*8kB 78*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873643] 1433 total pagecache pages
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873644] 0 pages in swap cache
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873645] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873646] Free swap = 0kB
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.873647] Total swap = 0kB
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879110] 985072 pages RAM
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879112] 24998 pages reserved
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879114] 3791 pages shared
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879115] 953243 pages non-shared
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879117] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879125] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879129] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879133] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879136] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879139] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879142] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879146] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879149] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879153] [ 790] 101 790 63477 647 1 0 0 rsyslogd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879156] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879159] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879162] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879165] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879168] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879172] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879175] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879178] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879181] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879184] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879188] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879191] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879194] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879197] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879200] [13468] 0 13468 89584 2529 1 0 0 php5-fpm
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879203] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879206] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879209] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879213] [16186] 108 16186 6792 75 0 0 0 pickup
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879216] [ 4301] 108 4301 6792 70 1 0 0 scache
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879219] [ 6319] 1012 6319 810657 747375 0 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879222] [ 6384] 1012 6384 68580 5604 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879225] [ 6479] 1012 6479 68580 5605 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879228] [ 6666] 1012 6666 1099 25 1 0 0 sh
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879231] [ 6667] 1012 6667 68825 5870 1 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879234] [ 6673] 1012 6673 1099 25 1 0 0 sh
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879237] [ 6674] 1012 6674 4676 58 1 0 0 git
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879240] [ 6675] 1012 6675 68499 5551 0 0 0 php
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879243] [ 6677] 1012 6677 1099 24 1 0 0 sh
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879246] [ 6678] 1012 6678 8213 47 0 0 0 ssh
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879249] Out of memory: Kill process 6319 (php) score 780 or sacrifice child
Jul 16 04:49:09 ip-10-61-174-46 kernel: [1477044.879261] Killed process 6319 (php) total-vm:3242628kB, anon-rss:2989496kB, file-rss:4kB
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269246] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269251] php cpuset=/ mems_allowed=0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269254] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269256] Call Trace:
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269263] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269266] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269268] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269271] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269274] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269278] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269280] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269283] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269285] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269288] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269291] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269293] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269297] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269300] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269304] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269307] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269310] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269311] Mem-Info:
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269312] Node 0 DMA per-cpu:
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269314] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269316] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269317] Node 0 DMA32 per-cpu:
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269319] CPU 0: hi: 186, btch: 31 usd: 44
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269320] CPU 1: hi: 186, btch: 31 usd: 43
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269322] active_anon:932775 inactive_anon:1393 isolated_anon:0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269323] active_file:3 inactive_file:7 isolated_file:20
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269324] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269325] free:4048 slab_reclaimable:2792 slab_unreclaimable:2592
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269325] mapped:1384 shmem:1405 pagetables:3064 bounce:0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269327] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269333] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269336] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731100kB inactive_anon:5572kB active_file:12kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5536kB shmem:5620kB slab_reclaimable:11168kB slab_unreclaimable:10368kB kernel_stack:728kB pagetables:12256kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:150 all_unreclaimable? yes
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269343] lowmem_reserve[]: 0 0 0 0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269345] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269352] Node 0 DMA32: 1441*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269359] 1485 total pagecache pages
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269360] 0 pages in swap cache
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269361] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269362] Free swap = 0kB
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.269363] Total swap = 0kB
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274685] 985072 pages RAM
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274687] 24998 pages reserved
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274688] 3794 pages shared
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274689] 953271 pages non-shared
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274691] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274700] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274703] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274707] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274710] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274713] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274716] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274719] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274723] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274726] [ 790] 101 790 63477 647 0 0 0 rsyslogd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274730] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274733] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274736] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274739] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274742] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274745] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274748] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274751] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274754] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274758] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274761] [ 654] 1012 654 70351 5779 0 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274764] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274767] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274770] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274773] [13468] 0 13468 89584 2499 0 0 0 php5-fpm
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274776] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274779] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274782] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274786] [16186] 108 16186 6792 75 0 0 0 pickup
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274789] [ 6479] 1012 6479 828088 764778 0 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274792] [17935] 1012 17935 68580 5604 0 0 0 php
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274795] Out of memory: Kill process 6479 (php) score 798 or sacrifice child
Jul 16 05:01:28 ip-10-61-174-46 kernel: [1477784.274806] Killed process 6479 (php) total-vm:3312352kB, anon-rss:3059044kB, file-rss:68kB
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548136] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548140] php cpuset=/ mems_allowed=0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548143] Pid: 21576, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548145] Call Trace:
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548151] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548154] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548156] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548159] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548162] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548165] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548168] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548171] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548173] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548176] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548179] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548181] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548185] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548187] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548189] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548192] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548194] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548196] Mem-Info:
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548197] Node 0 DMA per-cpu:
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548199] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548200] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548201] Node 0 DMA32 per-cpu:
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548203] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548204] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548207] active_anon:932429 inactive_anon:1393 isolated_anon:0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548208] active_file:21 inactive_file:21 isolated_file:0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548208] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548209] free:4048 slab_reclaimable:2726 slab_unreclaimable:2641
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548210] mapped:1357 shmem:1405 pagetables:3338 bounce:0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548211] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548218] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548220] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729716kB inactive_anon:5572kB active_file:84kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10904kB slab_unreclaimable:10564kB kernel_stack:808kB pagetables:13352kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:270 all_unreclaimable? yes
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548227] lowmem_reserve[]: 0 0 0 0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548230] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548236] Node 0 DMA32: 1456*4kB 6*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7920kB
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548243] 1470 total pagecache pages
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548244] 0 pages in swap cache
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548245] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548246] Free swap = 0kB
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.548247] Total swap = 0kB
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553582] 985072 pages RAM
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553584] 24998 pages reserved
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553585] 3804 pages shared
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553586] 953306 pages non-shared
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553588] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553597] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553601] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553604] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553607] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553610] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553613] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553617] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553620] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553624] [ 790] 101 790 63477 647 0 0 0 rsyslogd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553627] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553630] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553634] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553637] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553640] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553643] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553646] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553649] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553652] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553656] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553659] [ 654] 1012 654 70351 5779 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553662] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553665] [ 658] 1012 658 199421 134839 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553668] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553671] [13468] 0 13468 89584 2514 0 0 0 php5-fpm
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553674] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553677] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553680] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553684] [20559] 108 20559 6792 69 0 0 0 pickup
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553687] [21576] 1012 21576 820623 757399 0 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553690] [ 3117] 108 3117 6818 73 1 0 0 cleanup
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553693] [ 3118] 108 3118 6795 70 0 0 0 trivial-rewrite
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553696] [ 3119] 108 3119 11107 158 0 0 0 smtp
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553699] [ 3123] 108 3123 11107 142 1 0 0 smtp
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553702] [ 3129] 108 3129 11107 158 1 0 0 smtp
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553705] [ 3135] 108 3135 11107 159 1 0 0 smtp
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553708] [ 3139] 108 3139 11107 158 1 0 0 smtp
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553711] [ 3572] 1012 3572 68580 5604 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553715] [ 3754] 1012 3754 1099 25 1 0 0 sh
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553718] [ 3755] 1012 3755 68982 5974 1 0 0 php
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553721] [ 3774] 1012 3774 1099 25 0 0 0 sh
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553724] [ 3775] 1012 3775 4647 48 1 0 0 git
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553726] Out of memory: Kill process 21576 (php) score 790 or sacrifice child
Jul 16 05:21:33 ip-10-61-174-46 kernel: [1478988.553738] Killed process 21576 (php) total-vm:3282492kB, anon-rss:3029508kB, file-rss:88kB
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192126] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192129] php cpuset=/ mems_allowed=0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192132] Pid: 3838, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192134] Call Trace:
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192141] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192143] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192145] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192148] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192152] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192155] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192157] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192161] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192164] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192167] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192169] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192171] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192174] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192176] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192179] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192182] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192184] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192187] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192189] Mem-Info:
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192190] Node 0 DMA per-cpu:
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192192] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192193] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192194] Node 0 DMA32 per-cpu:
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192196] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192197] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192199] active_anon:932745 inactive_anon:1393 isolated_anon:0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192200] active_file:0 inactive_file:0 isolated_file:0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192201] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192201] free:4048 slab_reclaimable:2631 slab_unreclaimable:2650
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192202] mapped:1354 shmem:1405 pagetables:3213 bounce:0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192204] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192210] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192213] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730980kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:10524kB slab_unreclaimable:10600kB kernel_stack:768kB pagetables:12852kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192220] lowmem_reserve[]: 0 0 0 0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192222] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192229] Node 0 DMA32: 1222*4kB 123*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7920kB
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192235] 1445 total pagecache pages
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192237] 0 pages in swap cache
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192238] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192239] Free swap = 0kB
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.192240] Total swap = 0kB
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197733] 985072 pages RAM
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197736] 24998 pages reserved
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197737] 3814 pages shared
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197738] 953304 pages non-shared
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197740] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197748] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197752] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197755] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197758] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197761] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197765] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197768] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197771] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197775] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197778] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197781] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197785] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197788] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197791] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197794] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197797] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197800] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197803] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197806] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197810] [ 654] 1012 654 70351 5780 0 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197813] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197816] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197819] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197822] [13468] 0 13468 89584 2499 0 0 0 php5-fpm
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197825] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197828] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197831] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197834] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197838] [ 3838] 1012 3838 823627 760415 0 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197841] [ 7586] 108 7586 6818 73 0 0 0 cleanup
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197844] [ 7587] 108 7587 6795 71 0 0 0 trivial-rewrite
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197847] [ 7588] 108 7588 11107 159 1 0 0 smtp
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197850] [ 7592] 1012 7592 68580 5604 0 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197853] [ 7802] 1012 7802 1099 25 0 0 0 sh
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197857] [ 7803] 1012 7803 66418 3984 1 0 0 php
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197859] Out of memory: Kill process 3838 (php) score 793 or sacrifice child
Jul 16 05:25:53 ip-10-61-174-46 kernel: [1479249.197871] Killed process 3838 (php) total-vm:3294508kB, anon-rss:3041552kB, file-rss:108kB
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804119] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804122] php cpuset=/ mems_allowed=0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804125] Pid: 7838, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804127] Call Trace:
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804133] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804136] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804138] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804141] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804144] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804147] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804150] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804153] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804155] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804158] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804161] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804164] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804166] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804169] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804171] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804173] Mem-Info:
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804174] Node 0 DMA per-cpu:
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804176] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804178] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804179] Node 0 DMA32 per-cpu:
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804180] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804182] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804184] active_anon:932970 inactive_anon:1393 isolated_anon:0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804185] active_file:0 inactive_file:0 isolated_file:0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804185] unevictable:0 dirty:0 writeback:23 unstable:0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804186] free:4048 slab_reclaimable:2390 slab_unreclaimable:2679
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804187] mapped:1363 shmem:1405 pagetables:3209 bounce:0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804188] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804195] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804198] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731880kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:92kB mapped:5452kB shmem:5620kB slab_reclaimable:9560kB slab_unreclaimable:10716kB kernel_stack:776kB pagetables:12836kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804205] lowmem_reserve[]: 0 0 0 0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804207] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804214] Node 0 DMA32: 1225*4kB 108*8kB 4*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7940kB
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804221] 1417 total pagecache pages
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804222] 0 pages in swap cache
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804223] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804224] Free swap = 0kB
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.804225] Total swap = 0kB
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809641] 985072 pages RAM
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809644] 24998 pages reserved
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809645] 3772 pages shared
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809646] 953338 pages non-shared
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809648] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809657] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809660] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809664] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809667] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809670] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809673] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809676] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809680] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809684] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809687] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809690] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809694] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809697] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809700] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809703] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809707] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809710] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809713] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809716] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809719] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809722] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809725] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809728] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809731] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809734] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809737] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809740] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809744] [20559] 108 20559 6792 75 1 0 0 pickup
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809747] [ 7838] 1012 7838 827471 764134 0 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809750] [11423] 108 11423 6818 74 1 0 0 cleanup
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809754] [11424] 108 11424 6795 70 0 0 0 trivial-rewrite
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809757] [11425] 108 11425 11107 157 0 0 0 smtp
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809760] [11429] 108 11429 11107 158 0 0 0 smtp
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809763] [11433] 108 11433 11107 159 1 0 0 smtp
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809766] [11437] 108 11437 11107 158 0 0 0 smtp
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809769] [12725] 1012 12725 68580 5604 1 0 0 php
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809772] Out of memory: Kill process 7838 (php) score 797 or sacrifice child
Jul 16 05:31:28 ip-10-61-174-46 kernel: [1479583.809783] Killed process 7838 (php) total-vm:3309884kB, anon-rss:3056532kB, file-rss:4kB
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488121] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488125] php cpuset=/ mems_allowed=0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488128] Pid: 13843, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488129] Call Trace:
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488136] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488138] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488140] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488143] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488147] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488150] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488152] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488156] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488159] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488162] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488164] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488166] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488171] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488173] [<ffffffff8113f9d1>] ? handle_mm_fault+0x1/0x370
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488177] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488179] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488182] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488185] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488186] Mem-Info:
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488187] Node 0 DMA per-cpu:
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488189] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488191] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488192] Node 0 DMA32 per-cpu:
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488193] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488195] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488197] active_anon:932988 inactive_anon:1393 isolated_anon:0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488198] active_file:0 inactive_file:0 isolated_file:0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488198] unevictable:0 dirty:30 writeback:0 unstable:0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488199] free:4048 slab_reclaimable:2642 slab_unreclaimable:2602
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488200] mapped:1379 shmem:1405 pagetables:3072 bounce:0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488201] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488208] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488211] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731952kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:120kB writeback:0kB mapped:5516kB shmem:5620kB slab_reclaimable:10568kB slab_unreclaimable:10408kB kernel_stack:728kB pagetables:12288kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488218] lowmem_reserve[]: 0 0 0 0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488220] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488227] Node 0 DMA32: 847*4kB 239*8kB 26*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7860kB
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488233] 1472 total pagecache pages
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488235] 0 pages in swap cache
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488236] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488237] Free swap = 0kB
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.488238] Total swap = 0kB
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493643] 985072 pages RAM
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493646] 24998 pages reserved
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493647] 3876 pages shared
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493648] 953311 pages non-shared
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493650] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493659] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493662] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493665] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493668] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493672] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493675] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493678] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493682] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493686] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493689] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493692] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493695] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493698] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493701] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493705] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493708] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493711] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493714] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493717] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493721] [ 654] 1012 654 70351 5779 1 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493724] [ 657] 1012 657 69242 6263 1 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493727] [ 658] 1012 658 199421 134845 0 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493730] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493733] [13468] 0 13468 89584 2518 1 0 0 php5-fpm
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493736] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493739] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493742] [ 2686] 1012 2686 68618 5680 0 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493746] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493749] [13843] 1012 13843 828159 764951 0 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493752] [14696] 1012 14696 68580 5603 1 0 0 php
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493755] Out of memory: Kill process 13843 (php) score 798 or sacrifice child
Jul 16 05:33:44 ip-10-61-174-46 kernel: [1479720.493766] Killed process 13843 (php) total-vm:3312636kB, anon-rss:3059704kB, file-rss:100kB
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144142] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144146] php cpuset=/ mems_allowed=0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144149] Pid: 15004, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144150] Call Trace:
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144159] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144161] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144164] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144168] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144171] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144174] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144177] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144179] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144181] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144184] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144186] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144189] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144192] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144194] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144196] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144199] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144202] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144203] Mem-Info:
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144204] Node 0 DMA per-cpu:
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144206] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144208] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144209] Node 0 DMA32 per-cpu:
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144210] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144212] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144214] active_anon:933013 inactive_anon:1393 isolated_anon:0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144215] active_file:10 inactive_file:13 isolated_file:0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144216] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144216] free:4048 slab_reclaimable:2576 slab_unreclaimable:2621
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144217] mapped:1360 shmem:1405 pagetables:3069 bounce:0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144218] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144225] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144228] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732052kB inactive_anon:5572kB active_file:40kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:10304kB slab_unreclaimable:10484kB kernel_stack:728kB pagetables:12276kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:17212 all_unreclaimable? yes
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144235] lowmem_reserve[]: 0 0 0 0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144237] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144244] Node 0 DMA32: 1161*4kB 153*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7916kB
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144251] 1492 total pagecache pages
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144252] 0 pages in swap cache
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144253] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144254] Free swap = 0kB
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.144255] Total swap = 0kB
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149673] 985072 pages RAM
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149676] 24998 pages reserved
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149677] 3805 pages shared
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149678] 953317 pages non-shared
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149680] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149689] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149692] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149695] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149698] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149702] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149705] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149708] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149711] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149715] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149718] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149722] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149725] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149728] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149731] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149734] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149737] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149740] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149743] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149746] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149749] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149753] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149756] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149759] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149762] [13468] 0 13468 89584 2501 1 0 0 php5-fpm
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149765] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149768] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149771] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149774] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149778] [15004] 1012 15004 828194 764943 0 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149781] [16021] 1012 16021 68580 5605 1 0 0 php
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149784] Out of memory: Kill process 15004 (php) score 798 or sacrifice child
Jul 16 05:35:20 ip-10-61-174-46 kernel: [1479816.149795] Killed process 15004 (php) total-vm:3312776kB, anon-rss:3059720kB, file-rss:52kB
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128147] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128151] php cpuset=/ mems_allowed=0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128154] Pid: 16324, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128155] Call Trace:
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128162] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128165] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128167] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128170] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128173] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128176] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128182] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128184] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128195] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128198] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128203] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128207] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128209] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128212] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128215] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128216] Mem-Info:
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128217] Node 0 DMA per-cpu:
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128219] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128221] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128222] Node 0 DMA32 per-cpu:
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128223] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128225] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128227] active_anon:932977 inactive_anon:1393 isolated_anon:0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128228] active_file:0 inactive_file:0 isolated_file:0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128228] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128229] free:4048 slab_reclaimable:2587 slab_unreclaimable:2593
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128230] mapped:1357 shmem:1405 pagetables:3075 bounce:0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128231] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128238] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128241] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731908kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10348kB slab_unreclaimable:10372kB kernel_stack:728kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128247] lowmem_reserve[]: 0 0 0 0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128250] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128256] Node 0 DMA32: 938*4kB 235*8kB 9*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128263] 1485 total pagecache pages
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128264] 0 pages in swap cache
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128265] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128266] Free swap = 0kB
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.128267] Total swap = 0kB
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133682] 985072 pages RAM
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133685] 24998 pages reserved
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133686] 3824 pages shared
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133687] 953317 pages non-shared
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133689] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133698] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133702] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133705] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133708] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133711] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133714] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133717] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133721] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133725] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133728] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133731] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133734] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133737] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133740] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133743] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133747] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133750] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133753] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133756] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133759] [ 654] 1012 654 70351 5779 1 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133762] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133765] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133768] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133771] [13468] 0 13468 89584 2486 0 0 0 php5-fpm
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133774] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133777] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133780] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133784] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133787] [16324] 1012 16324 828194 764931 0 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133790] [20056] 1012 20056 68580 5605 0 0 0 php
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133793] Out of memory: Kill process 16324 (php) score 798 or sacrifice child
Jul 16 05:39:49 ip-10-61-174-46 kernel: [1480085.133804] Killed process 16324 (php) total-vm:3312776kB, anon-rss:3059716kB, file-rss:8kB
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416331] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416336] ntpd cpuset=/ mems_allowed=0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416340] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416342] Call Trace:
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416349] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416353] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416356] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416360] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416366] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416370] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416375] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416377] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416381] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416385] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416389] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416392] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416395] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416400] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416403] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416405] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416410] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416412] Mem-Info:
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416413] Node 0 DMA per-cpu:
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416415] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416417] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416419] Node 0 DMA32 per-cpu:
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416421] CPU 0: hi: 186, btch: 31 usd: 1
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416423] CPU 1: hi: 186, btch: 31 usd: 28
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416426] active_anon:932046 inactive_anon:1393 isolated_anon:0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416427] active_file:1 inactive_file:24 isolated_file:0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416428] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416429] free:4047 slab_reclaimable:2461 slab_unreclaimable:2671
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416430] mapped:1370 shmem:1405 pagetables:3836 bounce:0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416432] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416441] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416445] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3728184kB inactive_anon:5572kB active_file:4kB inactive_file:96kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:9844kB slab_unreclaimable:10684kB kernel_stack:896kB pagetables:15344kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:131 all_unreclaimable? yes
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416453] lowmem_reserve[]: 0 0 0 0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416457] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416466] Node 0 DMA32: 964*4kB 241*8kB 11*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8008kB
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416476] 1442 total pagecache pages
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416477] 0 pages in swap cache
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416479] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416481] Free swap = 0kB
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.416482] Total swap = 0kB
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422508] 985072 pages RAM
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422510] 24998 pages reserved
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422512] 3780 pages shared
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422513] 953268 pages non-shared
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422515] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422524] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422527] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422530] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422533] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422535] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422538] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422541] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422544] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422548] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422551] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422554] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422557] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422561] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422563] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422567] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422569] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422572] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422575] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422578] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422581] [ 654] 1012 654 70351 5779 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422583] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422586] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422589] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422593] [13468] 0 13468 89584 2534 0 0 0 php5-fpm
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422596] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422599] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422602] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422605] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422609] [20405] 1012 20405 791629 728423 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422612] [24253] 1012 24253 73368 10480 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422615] [24367] 108 24367 6818 73 0 0 0 cleanup
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422618] [24368] 108 24368 6795 70 0 0 0 trivial-rewrite
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422621] [24369] 108 24369 11107 158 1 0 0 smtp
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422624] [24373] 108 24373 11107 158 1 0 0 smtp
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422627] [24377] 108 24377 11107 159 1 0 0 smtp
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422630] [24381] 108 24381 11107 159 1 0 0 smtp
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422633] [24469] 1012 24469 1099 24 1 0 0 sh
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422636] [24471] 1012 24471 68825 5870 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422639] [24473] 1012 24473 1099 25 1 0 0 sh
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422642] [24475] 1012 24475 68825 5873 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422645] [24476] 1012 24476 1099 24 1 0 0 sh
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422648] [24477] 1012 24477 68825 5869 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422651] [24488] 1012 24488 1099 26 1 0 0 sh
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422654] [24491] 1012 24491 4669 58 1 0 0 git
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422657] [24492] 1012 24492 68142 5226 1 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422660] [24496] 1012 24496 1099 26 1 0 0 sh
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422663] [24497] 1012 24497 5216 68 1 0 0 git
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422666] [24498] 1012 24498 65963 3552 0 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422669] [24499] 1012 24499 1099 26 1 0 0 sh
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422671] [24500] 1012 24500 4724 58 1 0 0 git
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422674] [24501] 1012 24501 65637 3199 1 0 0 php
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422677] Out of memory: Kill process 20405 (php) score 760 or sacrifice child
Jul 16 05:44:14 ip-10-61-174-46 kernel: [1480350.422688] Killed process 20405 (php) total-vm:3166516kB, anon-rss:2913692kB, file-rss:0kB
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392132] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392136] php cpuset=/ mems_allowed=0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392139] Pid: 24592, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392140] Call Trace:
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392147] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392150] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392152] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392155] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392158] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392162] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392164] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392167] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392170] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392172] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392175] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392178] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392180] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392183] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392186] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392189] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392192] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392193] Mem-Info:
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392194] Node 0 DMA per-cpu:
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392196] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392197] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392198] Node 0 DMA32 per-cpu:
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392200] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392201] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392204] active_anon:932766 inactive_anon:1393 isolated_anon:0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392205] active_file:15 inactive_file:21 isolated_file:0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392205] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392206] free:4048 slab_reclaimable:2587 slab_unreclaimable:2629
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392207] mapped:1375 shmem:1405 pagetables:3223 bounce:0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392208] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392215] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392218] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731064kB inactive_anon:5572kB active_file:60kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5500kB shmem:5620kB slab_reclaimable:10348kB slab_unreclaimable:10516kB kernel_stack:784kB pagetables:12892kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10728 all_unreclaimable? yes
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392224] lowmem_reserve[]: 0 0 0 0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392227] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392234] Node 0 DMA32: 773*4kB 211*8kB 68*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392240] 1437 total pagecache pages
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392242] 0 pages in swap cache
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392243] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392244] Free swap = 0kB
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.392245] Total swap = 0kB
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397797] 985072 pages RAM
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397799] 24998 pages reserved
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397801] 3854 pages shared
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397802] 953313 pages non-shared
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397804] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397812] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397816] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397819] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397822] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397825] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397828] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397831] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397835] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397839] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397841] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397844] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397848] [ 975] 0 975 6276 81 0 0 0 master
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397851] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397854] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397857] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397860] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397863] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397866] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397870] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397873] [ 654] 1012 654 70351 5784 1 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397876] [ 657] 1012 657 69242 6262 1 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397878] [ 658] 1012 658 199421 134844 1 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397882] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397885] [13468] 0 13468 89584 2520 1 0 0 php5-fpm
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397888] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397891] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397894] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397898] [20559] 108 20559 6792 75 1 0 0 pickup
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397901] [24367] 108 24367 6818 81 1 0 0 cleanup
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397905] [24368] 108 24368 6795 74 0 0 0 trivial-rewrite
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397908] [24369] 108 24369 11107 157 0 0 0 smtp
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397911] [24373] 108 24373 11107 161 0 0 0 smtp
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397914] [24377] 108 24377 11107 162 0 0 0 smtp
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397917] [24381] 108 24381 11107 158 0 0 0 smtp
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397919] [24592] 1012 24592 827109 763852 0 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397922] [25081] 108 25081 6792 68 1 0 0 scache
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397926] [26065] 1012 26065 68580 5608 1 0 0 php
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397928] Out of memory: Kill process 24592 (php) score 797 or sacrifice child
Jul 16 05:46:15 ip-10-61-174-46 kernel: [1480471.397939] Killed process 24592 (php) total-vm:3308436kB, anon-rss:3055392kB, file-rss:16kB
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433269] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433273] php cpuset=/ mems_allowed=0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433276] Pid: 657, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433278] Call Trace:
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433284] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433287] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433289] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433292] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433297] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433299] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433303] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433305] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433308] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433310] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433313] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433315] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433317] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433320] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433323] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433325] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433328] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433330] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433333] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433336] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433339] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433343] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433346] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433349] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433351] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433353] Mem-Info:
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433354] Node 0 DMA per-cpu:
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433356] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433357] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433358] Node 0 DMA32 per-cpu:
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433360] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433361] CPU 1: hi: 186, btch: 31 usd: 36
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433364] active_anon:932837 inactive_anon:1393 isolated_anon:0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433365] active_file:7 inactive_file:0 isolated_file:0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433365] unevictable:0 dirty:29 writeback:0 unstable:0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433366] free:4039 slab_reclaimable:2635 slab_unreclaimable:2612
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433367] mapped:1354 shmem:1405 pagetables:3162 bounce:0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433368] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433375] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433378] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3731348kB inactive_anon:5572kB active_file:28kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:116kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:10540kB slab_unreclaimable:10448kB kernel_stack:760kB pagetables:12648kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:189 all_unreclaimable? yes
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433384] lowmem_reserve[]: 0 0 0 0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433387] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433393] Node 0 DMA32: 1149*4kB 147*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433400] 1404 total pagecache pages
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433401] 0 pages in swap cache
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433403] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433404] Free swap = 0kB
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.433404] Total swap = 0kB
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438779] 985072 pages RAM
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438782] 24998 pages reserved
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438783] 3770 pages shared
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438784] 953306 pages non-shared
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438786] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438795] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438798] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438802] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438805] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438808] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438811] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438814] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438818] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438822] [ 790] 101 790 63477 648 1 0 0 rsyslogd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438825] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438828] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438832] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438835] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438838] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438841] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438844] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438847] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438850] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438854] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438857] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438860] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438863] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438866] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438869] [13468] 0 13468 89584 2510 0 0 0 php5-fpm
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438872] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438875] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438878] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438882] [20559] 108 20559 6792 75 1 0 0 pickup
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438885] [26370] 1012 26370 827559 764326 1 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438889] [30805] 108 30805 6818 73 1 0 0 cleanup
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438892] [30806] 108 30806 6795 70 1 0 0 trivial-rewrite
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438895] [30807] 108 30807 11107 158 1 0 0 smtp
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438898] [30811] 108 30811 11107 159 1 0 0 smtp
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438901] [31941] 1012 31941 68580 5604 1 0 0 php
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438904] Out of memory: Kill process 26370 (php) score 797 or sacrifice child
Jul 16 05:52:35 ip-10-61-174-46 kernel: [1480851.438914] Killed process 26370 (php) total-vm:3310236kB, anon-rss:3057304kB, file-rss:0kB
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824270] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824274] php cpuset=/ mems_allowed=0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824277] Pid: 32126, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824279] Call Trace:
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824285] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824288] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824290] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824293] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824298] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824300] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824304] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824306] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824309] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824312] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824315] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824318] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824320] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824323] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824326] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824328] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824330] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824332] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824334] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824337] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824339] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824341] Mem-Info:
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824342] Node 0 DMA per-cpu:
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824344] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824346] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824347] Node 0 DMA32 per-cpu:
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824348] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824350] CPU 1: hi: 186, btch: 31 usd: 52
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824352] active_anon:932863 inactive_anon:1393 isolated_anon:0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824353] active_file:12 inactive_file:24 isolated_file:0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824354] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824354] free:4037 slab_reclaimable:2346 slab_unreclaimable:2589
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824355] mapped:1359 shmem:1405 pagetables:3317 bounce:0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824356] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824363] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824366] Node 0 DMA32 free:7812kB min:7848kB low:9808kB high:11772kB active_anon:3731452kB inactive_anon:5572kB active_file:48kB inactive_file:96kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:9384kB slab_unreclaimable:10356kB kernel_stack:784kB pagetables:13268kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:120 all_unreclaimable? yes
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824372] lowmem_reserve[]: 0 0 0 0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824375] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824382] Node 0 DMA32: 754*4kB 210*8kB 51*16kB 6*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7816kB
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824388] 1465 total pagecache pages
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824389] 0 pages in swap cache
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824391] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824392] Free swap = 0kB
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.824393] Total swap = 0kB
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829712] 985072 pages RAM
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829714] 24998 pages reserved
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829715] 3787 pages shared
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829716] 953271 pages non-shared
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829718] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829727] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829730] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829733] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829736] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829740] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829743] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829746] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829749] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829753] [ 790] 101 790 63477 648 0 0 0 rsyslogd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829756] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829759] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829763] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829766] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829769] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829772] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829775] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829778] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829781] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829785] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829788] [ 654] 1012 654 70351 5779 1 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829791] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829794] [ 658] 1012 658 199421 134839 1 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829797] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829800] [13468] 0 13468 89584 2506 0 0 0 php5-fpm
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829803] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829806] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829809] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829813] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829816] [32126] 1012 32126 816290 753076 0 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829819] [ 425] 1012 425 68580 5604 1 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829822] [ 615] 1012 615 1099 25 0 0 0 sh
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829825] [ 616] 1012 616 68825 5871 0 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829828] [ 622] 1012 622 1099 25 0 0 0 sh
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829831] [ 623] 1012 623 4669 64 1 0 0 git
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829834] [ 624] 1012 624 68499 5552 0 0 0 php
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829837] [ 626] 1012 626 1099 25 0 0 0 sh
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829840] [ 627] 1012 627 10901 175 0 0 0 ssh
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829843] Out of memory: Kill process 32126 (php) score 786 or sacrifice child
Jul 16 05:53:44 ip-10-61-174-46 kernel: [1480919.829854] Killed process 32126 (php) total-vm:3265160kB, anon-rss:3012304kB, file-rss:0kB
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116009] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116014] ntpd cpuset=/ mems_allowed=0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116017] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116019] Call Trace:
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116027] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116030] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116034] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116038] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116042] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116047] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116052] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116067] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116070] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116075] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116078] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116081] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116085] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116089] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116094] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116096] Mem-Info:
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116097] Node 0 DMA per-cpu:
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116100] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116102] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116103] Node 0 DMA32 per-cpu:
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116105] CPU 0: hi: 186, btch: 31 usd: 79
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116107] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116111] active_anon:932262 inactive_anon:1393 isolated_anon:0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116112] active_file:2 inactive_file:22 isolated_file:56
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116113] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116114] free:4045 slab_reclaimable:2500 slab_unreclaimable:2653
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116115] mapped:1406 shmem:1405 pagetables:3551 bounce:0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116117] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116126] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116130] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3729048kB inactive_anon:5572kB active_file:8kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):224kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5624kB shmem:5620kB slab_reclaimable:10000kB slab_unreclaimable:10612kB kernel_stack:840kB pagetables:14204kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:242 all_unreclaimable? yes
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116138] lowmem_reserve[]: 0 0 0 0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116142] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116151] Node 0 DMA32: 781*4kB 196*8kB 23*16kB 13*32kB 3*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7844kB
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116161] 1478 total pagecache pages
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116162] 0 pages in swap cache
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116164] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116165] Free swap = 0kB
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.116167] Total swap = 0kB
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121816] 985072 pages RAM
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121818] 24998 pages reserved
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121819] 3775 pages shared
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121820] 953189 pages non-shared
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121822] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121829] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121832] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121834] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121836] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121838] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121840] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121843] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121845] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121848] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121850] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121852] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121855] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121857] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121859] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121861] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121863] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121866] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121868] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121870] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121872] [ 654] 1012 654 70351 5779 0 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121874] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121876] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121878] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121880] [13468] 0 13468 89584 2533 0 0 0 php5-fpm
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121883] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121885] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121887] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121890] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121892] [ 649] 1012 649 804001 740784 0 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121894] [ 2751] 1012 2751 68580 5604 1 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121896] [ 2909] 1012 2909 1099 24 0 0 0 sh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121898] [ 2910] 1012 2910 1099 24 0 0 0 sh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121901] [ 2911] 1012 2911 68825 5872 1 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121903] [ 2912] 1012 2912 68825 5871 1 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121905] [ 2923] 1012 2923 1099 25 0 0 0 sh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121907] [ 2924] 1012 2924 1099 25 0 0 0 sh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121909] [ 2925] 1012 2925 4676 58 0 0 0 git
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121911] [ 2926] 1012 2926 4776 58 0 0 0 git
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121913] [ 2927] 1012 2927 68499 5551 1 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121915] [ 2928] 1012 2928 68499 5552 0 0 0 php
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121917] [ 2931] 1012 2931 1099 24 0 0 0 sh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121919] [ 2932] 1012 2932 10901 176 0 0 0 ssh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121921] [ 2933] 1012 2933 1099 25 0 0 0 sh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121923] [ 2934] 1012 2934 10901 175 0 0 0 ssh
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121925] Out of memory: Kill process 649 (php) score 773 or sacrifice child
Jul 16 05:56:25 ip-10-61-174-46 kernel: [1481081.121934] Killed process 649 (php) total-vm:3216004kB, anon-rss:2963136kB, file-rss:0kB
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612122] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612126] php cpuset=/ mems_allowed=0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612129] Pid: 3033, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612130] Call Trace:
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612136] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612139] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612141] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612144] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612147] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612150] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612153] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612156] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612158] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612161] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612164] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612167] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612169] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612172] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612174] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612176] Mem-Info:
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612177] Node 0 DMA per-cpu:
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612179] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612180] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612181] Node 0 DMA32 per-cpu:
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612183] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612184] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612187] active_anon:932900 inactive_anon:1393 isolated_anon:0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612187] active_file:0 inactive_file:10 isolated_file:0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612188] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612189] free:4048 slab_reclaimable:2765 slab_unreclaimable:2581
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612189] mapped:1360 shmem:1405 pagetables:3056 bounce:0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612191] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612197] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612200] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731600kB inactive_anon:5572kB active_file:0kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:11060kB slab_unreclaimable:10324kB kernel_stack:728kB pagetables:12224kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5979 all_unreclaimable? yes
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612207] lowmem_reserve[]: 0 0 0 0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612209] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612216] Node 0 DMA32: 755*4kB 222*8kB 47*16kB 8*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7916kB
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612223] 1438 total pagecache pages
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612224] 0 pages in swap cache
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612225] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612226] Free swap = 0kB
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.612227] Total swap = 0kB
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617753] 985072 pages RAM
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617756] 24998 pages reserved
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617757] 3788 pages shared
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617759] 953334 pages non-shared
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617760] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617769] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617773] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617776] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617779] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617782] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617785] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617789] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617792] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617796] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617799] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617802] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617805] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617808] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617811] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617814] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617817] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617820] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617823] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617827] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617830] [ 654] 1012 654 70351 5779 1 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617833] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617836] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617839] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617842] [13468] 0 13468 89584 2518 1 0 0 php5-fpm
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617845] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617848] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617851] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617854] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617858] [ 3033] 1012 3033 828081 764859 0 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617861] [ 6538] 1012 6538 68580 5605 1 0 0 php
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617864] Out of memory: Kill process 3033 (php) score 798 or sacrifice child
Jul 16 06:00:59 ip-10-61-174-46 kernel: [1481354.617876] Killed process 3033 (php) total-vm:3312324kB, anon-rss:3059392kB, file-rss:44kB
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068141] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068145] php cpuset=/ mems_allowed=0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068148] Pid: 6864, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068149] Call Trace:
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068155] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068158] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068160] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068170] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068172] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068175] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068178] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068180] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068183] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068186] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068188] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068191] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068193] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068196] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068199] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068201] [<ffffffff811425e0>] ? find_vma+0x30/0x70
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068204] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068206] Mem-Info:
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068207] Node 0 DMA per-cpu:
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068209] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068210] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068211] Node 0 DMA32 per-cpu:
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068213] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068214] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068217] active_anon:932816 inactive_anon:1393 isolated_anon:0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068218] active_file:4 inactive_file:16 isolated_file:0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068218] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068219] free:4048 slab_reclaimable:2642 slab_unreclaimable:2610
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068220] mapped:1361 shmem:1405 pagetables:3190 bounce:0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068221] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068228] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068231] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731264kB inactive_anon:5572kB active_file:16kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5620kB slab_reclaimable:10568kB slab_unreclaimable:10440kB kernel_stack:752kB pagetables:12760kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12274 all_unreclaimable? yes
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068237] lowmem_reserve[]: 0 0 0 0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068240] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068246] Node 0 DMA32: 859*4kB 237*8kB 32*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7956kB
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068253] 1448 total pagecache pages
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068254] 0 pages in swap cache
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068256] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068257] Free swap = 0kB
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.068257] Total swap = 0kB
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073804] 985072 pages RAM
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073806] 24998 pages reserved
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073807] 3798 pages shared
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073808] 953312 pages non-shared
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073810] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073819] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073822] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073826] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073829] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073832] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073835] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073838] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073842] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073846] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073849] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073852] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073855] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073858] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073861] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073865] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073868] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073871] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073874] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073877] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073880] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073884] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073887] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073890] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073893] [13468] 0 13468 89584 2503 1 0 0 php5-fpm
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073896] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073899] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073902] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073906] [20559] 108 20559 6792 75 0 0 0 pickup
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073909] [ 6864] 1012 6864 821970 758755 0 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073912] [11672] 1012 11672 68580 5606 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073915] [11913] 1012 11913 1099 25 1 0 0 sh
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073918] [11914] 1012 11914 68982 6004 1 0 0 php
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073921] Out of memory: Kill process 6864 (php) score 791 or sacrifice child
Jul 16 06:06:50 ip-10-61-174-46 kernel: [1481706.073932] Killed process 6864 (php) total-vm:3287880kB, anon-rss:3035020kB, file-rss:0kB
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360144] php cpuset=/ mems_allowed=0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360147] Pid: 12005, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360148] Call Trace:
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360155] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360157] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360159] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360162] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360167] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360174] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360176] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360179] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360182] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360184] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360187] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360195] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360198] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360200] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360203] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360205] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360207] Mem-Info:
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360208] Node 0 DMA per-cpu:
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360210] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360211] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360213] Node 0 DMA32 per-cpu:
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360214] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360215] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360218] active_anon:933200 inactive_anon:1393 isolated_anon:0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360219] active_file:3 inactive_file:19 isolated_file:0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360219] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360220] free:4048 slab_reclaimable:2429 slab_unreclaimable:2616
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360221] mapped:1350 shmem:1405 pagetables:3109 bounce:0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360222] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360229] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360232] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732800kB inactive_anon:5572kB active_file:12kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:9716kB slab_unreclaimable:10464kB kernel_stack:792kB pagetables:12436kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:21097 all_unreclaimable? yes
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360238] lowmem_reserve[]: 0 0 0 0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360241] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360248] Node 0 DMA32: 1150*4kB 158*8kB 2*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8008kB
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360254] 1445 total pagecache pages
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360255] 0 pages in swap cache
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360257] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360258] Free swap = 0kB
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.360259] Total swap = 0kB
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365830] 985072 pages RAM
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365832] 24998 pages reserved
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365833] 3780 pages shared
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365835] 953319 pages non-shared
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365837] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365845] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365849] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365852] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365855] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365858] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365861] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365864] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365868] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365872] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365875] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365878] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365881] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365884] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365887] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365890] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365893] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365897] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365900] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365903] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365906] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365909] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365912] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365915] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365919] [13468] 0 13468 89584 2488 1 0 0 php5-fpm
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365922] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365924] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365928] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365931] [20559] 108 20559 6792 75 1 0 0 pickup
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365935] [12005] 1012 12005 828450 765180 0 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365938] [13357] 1012 13357 68580 5605 1 0 0 php
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365941] Out of memory: Kill process 12005 (php) score 798 or sacrifice child
Jul 16 06:08:44 ip-10-61-174-46 kernel: [1481820.365952] Killed process 12005 (php) total-vm:3313800kB, anon-rss:3060720kB, file-rss:0kB
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804349] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804354] collectd cpuset=/ mems_allowed=0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804357] Pid: 1051, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804359] Call Trace:
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804367] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804370] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804373] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804377] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804381] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804386] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804389] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804393] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804396] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804400] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804403] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804406] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804409] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804414] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804418] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804422] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804426] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804430] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804433] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804437] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804439] Mem-Info:
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804441] Node 0 DMA per-cpu:
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804443] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804445] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804447] Node 0 DMA32 per-cpu:
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804449] CPU 0: hi: 186, btch: 31 usd: 27
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804451] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804454] active_anon:932883 inactive_anon:1393 isolated_anon:0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804455] active_file:2 inactive_file:5 isolated_file:0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804456] unevictable:0 dirty:0 writeback:10 unstable:0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804457] free:4062 slab_reclaimable:2480 slab_unreclaimable:2606
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804458] mapped:1368 shmem:1405 pagetables:3246 bounce:0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804460] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804468] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804472] Node 0 DMA32 free:7912kB min:7848kB low:9808kB high:11772kB active_anon:3731532kB inactive_anon:5572kB active_file:8kB inactive_file:20kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:40kB mapped:5472kB shmem:5620kB slab_reclaimable:9920kB slab_unreclaimable:10424kB kernel_stack:768kB pagetables:12984kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:43 all_unreclaimable? yes
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804481] lowmem_reserve[]: 0 0 0 0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804484] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804502] Node 0 DMA32: 1161*4kB 141*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804510] 1426 total pagecache pages
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804512] 0 pages in swap cache
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804514] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804515] Free swap = 0kB
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.804516] Total swap = 0kB
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.809996] 985072 pages RAM
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.809999] 24998 pages reserved
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810000] 3785 pages shared
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810001] 953323 pages non-shared
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810003] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810011] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810015] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810018] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810021] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810024] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810028] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810031] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810034] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810039] [ 790] 101 790 63477 649 1 0 0 rsyslogd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810042] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810045] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810048] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810051] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810054] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810057] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810060] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810063] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810066] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810070] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810073] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810076] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810079] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810082] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810085] [13468] 0 13468 89584 2530 0 0 0 php5-fpm
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810088] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810091] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810094] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810098] [13666] 1012 13666 821711 758486 0 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810101] [19318] 108 19318 6792 69 1 0 0 pickup
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810104] [22186] 1012 22186 68580 5604 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810107] [22371] 1012 22371 1099 25 1 0 0 sh
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810110] [22372] 1012 22372 68825 5870 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810113] [22378] 1012 22378 1099 25 1 0 0 sh
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810116] [22379] 1012 22379 4676 57 1 0 0 git
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810119] [22380] 1012 22380 31456 382 1 0 0 php
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810122] Out of memory: Kill process 13666 (php) score 791 or sacrifice child
Jul 16 06:19:13 ip-10-61-174-46 kernel: [1482448.810132] Killed process 13666 (php) total-vm:3286844kB, anon-rss:3033944kB, file-rss:0kB
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828106] ssh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828110] ssh cpuset=/ mems_allowed=0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828113] Pid: 24636, comm: ssh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828115] Call Trace:
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828122] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828124] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828127] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828130] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828135] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828138] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828141] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828143] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828146] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828150] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828153] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828156] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828158] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828161] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828163] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828166] [<ffffffff81144d28>] ? do_mmap_pgoff+0x348/0x360
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828168] [<ffffffff81144e95>] ? sys_mmap_pgoff+0x155/0x230
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828171] [<ffffffff81146740>] ? sys_mprotect+0x1f0/0x250
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828174] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828176] Mem-Info:
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828177] Node 0 DMA per-cpu:
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828179] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828180] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828181] Node 0 DMA32 per-cpu:
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828183] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828184] CPU 1: hi: 186, btch: 31 usd: 1
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828187] active_anon:932898 inactive_anon:1393 isolated_anon:0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828187] active_file:0 inactive_file:25 isolated_file:0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828188] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828189] free:4057 slab_reclaimable:2387 slab_unreclaimable:2598
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828189] mapped:1353 shmem:1405 pagetables:3313 bounce:0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828191] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828198] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828200] Node 0 DMA32 free:7892kB min:7848kB low:9808kB high:11772kB active_anon:3731592kB inactive_anon:5572kB active_file:0kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:9548kB slab_unreclaimable:10392kB kernel_stack:784kB pagetables:13252kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:123 all_unreclaimable? yes
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828207] lowmem_reserve[]: 0 0 0 0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828209] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828216] Node 0 DMA32: 889*4kB 216*8kB 27*16kB 2*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7892kB
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828223] 1431 total pagecache pages
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828224] 0 pages in swap cache
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828225] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828227] Free swap = 0kB
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.828227] Total swap = 0kB
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833953] 985072 pages RAM
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833955] 24998 pages reserved
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833957] 3786 pages shared
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833958] 953321 pages non-shared
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833960] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833968] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833972] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833975] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833978] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833981] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833985] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833988] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833991] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833996] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.833999] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834002] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834005] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834008] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834011] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834015] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834018] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834021] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834024] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834027] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834030] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834033] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834036] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834039] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834042] [13468] 0 13468 89584 2501 1 0 0 php5-fpm
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834045] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834048] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834051] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834055] [19318] 108 19318 6792 69 0 0 0 pickup
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834058] [22438] 1012 22438 816482 753248 0 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834062] [24435] 1012 24435 68580 5604 0 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834065] [24624] 1012 24624 1099 25 1 0 0 sh
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834068] [24625] 1012 24625 68825 5871 1 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834071] [24631] 1012 24631 1099 25 0 0 0 sh
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834074] [24632] 1012 24632 4776 58 1 0 0 git
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834077] [24633] 1012 24633 68499 5552 1 0 0 php
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834080] [24635] 1012 24635 1099 25 0 0 0 sh
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834083] [24636] 1012 24636 8209 39 1 0 0 ssh
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834086] Out of memory: Kill process 22438 (php) score 786 or sacrifice child
Jul 16 06:22:06 ip-10-61-174-46 kernel: [1482621.834096] Killed process 22438 (php) total-vm:3265928kB, anon-rss:3012992kB, file-rss:0kB
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480143] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480147] php cpuset=/ mems_allowed=0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480150] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480151] Call Trace:
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480158] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480170] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480173] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480177] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480182] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480184] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480195] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480198] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480204] [<ffffffff8129f083>] ? security_file_permission+0x93/0xb0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480207] [<ffffffff81179501>] ? rw_verify_area+0x61/0xf0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480209] [<ffffffff811799e0>] ? vfs_read+0xb0/0x180
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480212] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480214] Mem-Info:
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480215] Node 0 DMA per-cpu:
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480217] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480218] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480219] Node 0 DMA32 per-cpu:
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480221] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480222] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480225] active_anon:933086 inactive_anon:1393 isolated_anon:0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480225] active_file:0 inactive_file:26 isolated_file:0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480226] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480227] free:4048 slab_reclaimable:2527 slab_unreclaimable:2583
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480227] mapped:1357 shmem:1405 pagetables:3073 bounce:0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480229] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480235] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480238] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732344kB inactive_anon:5572kB active_file:0kB inactive_file:104kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10108kB slab_unreclaimable:10332kB kernel_stack:728kB pagetables:12292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3787 all_unreclaimable? yes
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480245] lowmem_reserve[]: 0 0 0 0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480247] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480254] Node 0 DMA32: 931*4kB 155*8kB 39*16kB 10*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8020kB
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480261] 1458 total pagecache pages
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480262] 0 pages in swap cache
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480263] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480264] Free swap = 0kB
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.480265] Total swap = 0kB
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485753] 985072 pages RAM
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485756] 24998 pages reserved
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485757] 3782 pages shared
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485758] 953315 pages non-shared
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485760] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485768] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485772] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485775] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485778] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485781] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485785] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485788] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485791] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485795] [ 790] 101 790 63477 649 1 0 0 rsyslogd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485798] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485801] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485805] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485808] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485811] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485814] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485817] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485820] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485823] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485826] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485830] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485832] [ 657] 1012 657 69242 6257 0 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485836] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485839] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485842] [13468] 0 13468 89584 2501 1 0 0 php5-fpm
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485845] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485848] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485851] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485855] [19318] 108 19318 6792 75 0 0 0 pickup
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485859] [24716] 1012 24716 828322 765054 1 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485862] [25247] 1012 25247 68580 5604 1 0 0 php
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485864] Out of memory: Kill process 24716 (php) score 798 or sacrifice child
Jul 16 06:22:59 ip-10-61-174-46 kernel: [1482675.485876] Killed process 24716 (php) total-vm:3313288kB, anon-rss:3060216kB, file-rss:0kB
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920121] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920125] php cpuset=/ mems_allowed=0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920129] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920131] Call Trace:
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920142] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920145] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920148] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920153] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920156] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920158] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920161] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920164] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920166] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920168] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920171] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920173] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920176] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920178] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920180] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920183] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920187] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920190] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920193] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920195] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920197] Mem-Info:
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920198] Node 0 DMA per-cpu:
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920200] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920201] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920202] Node 0 DMA32 per-cpu:
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920204] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920205] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920208] active_anon:932784 inactive_anon:1393 isolated_anon:0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920208] active_file:0 inactive_file:0 isolated_file:0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920209] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920210] free:4048 slab_reclaimable:2377 slab_unreclaimable:2656
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920210] mapped:1358 shmem:1405 pagetables:3283 bounce:0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920212] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920219] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920222] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731136kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5620kB slab_reclaimable:9508kB slab_unreclaimable:10624kB kernel_stack:832kB pagetables:13132kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920228] lowmem_reserve[]: 0 0 0 0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920230] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920237] Node 0 DMA32: 1092*4kB 178*8kB 7*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920244] 1479 total pagecache pages
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920245] 0 pages in swap cache
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920246] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920247] Free swap = 0kB
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.920248] Total swap = 0kB
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925704] 985072 pages RAM
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925706] 24998 pages reserved
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925708] 3818 pages shared
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925709] 953262 pages non-shared
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925711] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925719] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925723] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925726] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925729] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925732] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925736] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925739] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925742] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925746] [ 790] 101 790 63477 649 1 0 0 rsyslogd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925750] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925753] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925756] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925759] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925762] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925765] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925768] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925772] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925775] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925778] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925781] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925784] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925787] [ 658] 1012 658 199421 134839 0 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925790] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925793] [13468] 0 13468 89584 2502 1 0 0 php5-fpm
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925796] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925799] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925802] [ 2686] 1012 2686 68618 5677 0 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925806] [19318] 108 19318 6792 75 1 0 0 pickup
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925809] [25549] 1012 25549 827007 763690 0 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925812] [26231] 108 26231 6818 73 0 0 0 cleanup
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925815] [26232] 108 26232 6795 71 0 0 0 trivial-rewrite
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925819] [26233] 108 26233 11107 157 0 0 0 smtp
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925821] [26237] 108 26237 11107 157 0 0 0 smtp
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925824] [26241] 108 26241 11107 157 0 0 0 smtp
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925828] [27246] 0 27246 8460 79 0 0 0 cron
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925830] [27250] 0 27250 1099 25 1 0 0 sh
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925833] [27251] 0 27251 1074 26 0 0 0 run-parts
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925836] [27254] 0 27254 1099 36 0 0 0 apt
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925839] [27286] 0 27286 1469 23 0 0 0 sleep
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925842] [27570] 108 27570 6800 69 1 0 0 bounce
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925846] [27919] 1012 27919 68580 5604 1 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925849] [28120] 1012 28120 1099 25 1 0 0 sh
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925851] [28121] 1012 28121 12484 152 1 0 0 php
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925854] Out of memory: Kill process 25549 (php) score 797 or sacrifice child
Jul 16 06:26:04 ip-10-61-174-46 kernel: [1482859.925866] Killed process 25549 (php) total-vm:3308028kB, anon-rss:3054760kB, file-rss:0kB
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805131] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805136] php cpuset=/ mems_allowed=0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805140] Pid: 28224, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805142] Call Trace:
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805149] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805153] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805156] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805160] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805166] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805174] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805185] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805189] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805192] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805195] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805201] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805205] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805209] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805212] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805216] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805221] Mem-Info:
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805223] Node 0 DMA per-cpu:
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805226] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805227] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805229] Node 0 DMA32 per-cpu:
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805231] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805233] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805237] active_anon:932556 inactive_anon:1393 isolated_anon:0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805238] active_file:8 inactive_file:8 isolated_file:0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805239] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805240] free:4048 slab_reclaimable:2629 slab_unreclaimable:2623
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805241] mapped:1354 shmem:1405 pagetables:3367 bounce:0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805251] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805255] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730224kB inactive_anon:5572kB active_file:32kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:10516kB slab_unreclaimable:10492kB kernel_stack:824kB pagetables:13468kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:100 all_unreclaimable? yes
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805264] lowmem_reserve[]: 0 0 0 0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805267] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805277] Node 0 DMA32: 808*4kB 212*8kB 45*16kB 3*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805286] 1421 total pagecache pages
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805287] 0 pages in swap cache
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805289] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805290] Free swap = 0kB
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.805291] Total swap = 0kB
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.810986] 985072 pages RAM
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.810988] 24998 pages reserved
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.810989] 3803 pages shared
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.810991] 953322 pages non-shared
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.810993] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811001] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811004] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811008] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811011] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811014] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811017] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811020] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811024] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811028] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811031] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811034] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811037] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811040] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811043] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811046] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811049] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811053] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811056] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811059] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811062] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811065] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811068] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811071] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811074] [13468] 0 13468 89584 2531 1 0 0 php5-fpm
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811077] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811080] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811083] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811086] [19318] 108 19318 6792 75 0 0 0 pickup
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811089] [27246] 0 27246 8460 79 0 0 0 cron
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811092] [27250] 0 27250 1099 25 1 0 0 sh
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811095] [27251] 0 27251 1074 26 0 0 0 run-parts
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811098] [27254] 0 27254 1099 36 0 0 0 apt
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811101] [27286] 0 27286 1469 23 0 0 0 sleep
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811104] [28224] 1012 28224 815805 752540 0 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811107] [29880] 1012 29880 68580 5605 1 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811110] [30024] 1012 30024 1099 26 0 0 0 sh
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811113] [30025] 1012 30025 68825 5871 1 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811116] [30031] 1012 30031 1099 26 0 0 0 sh
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811119] [30032] 1012 30032 5216 68 0 0 0 git
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811122] [30033] 1012 30033 68499 5552 0 0 0 php
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811125] [30035] 1012 30035 1099 25 0 0 0 sh
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811128] [30036] 1012 30036 10901 175 1 0 0 ssh
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811130] Out of memory: Kill process 28224 (php) score 785 or sacrifice child
Jul 16 06:28:29 ip-10-61-174-46 kernel: [1483004.811141] Killed process 28224 (php) total-vm:3263220kB, anon-rss:3010156kB, file-rss:4kB
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309283] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309286] php cpuset=/ mems_allowed=0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309289] Pid: 30119, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309291] Call Trace:
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309298] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309300] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309302] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309305] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309310] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309313] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309317] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309319] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309322] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309324] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309327] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309330] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309332] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309335] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309337] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309341] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309343] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309345] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309348] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309350] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309352] Mem-Info:
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309353] Node 0 DMA per-cpu:
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309355] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309356] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309358] Node 0 DMA32 per-cpu:
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309359] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309361] CPU 1: hi: 186, btch: 31 usd: 32
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309363] active_anon:933126 inactive_anon:1393 isolated_anon:0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309364] active_file:17 inactive_file:0 isolated_file:0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309364] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309365] free:4020 slab_reclaimable:2299 slab_unreclaimable:2630
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309366] mapped:1368 shmem:1405 pagetables:3101 bounce:0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309367] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309374] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309377] Node 0 DMA32 free:7744kB min:7848kB low:9808kB high:11772kB active_anon:3732504kB inactive_anon:5572kB active_file:68kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5472kB shmem:5620kB slab_reclaimable:9196kB slab_unreclaimable:10520kB kernel_stack:768kB pagetables:12404kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:179 all_unreclaimable? yes
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309384] lowmem_reserve[]: 0 0 0 0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309386] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309393] Node 0 DMA32: 790*4kB 197*8kB 60*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7744kB
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309400] 1431 total pagecache pages
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309401] 0 pages in swap cache
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309402] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309403] Free swap = 0kB
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.309404] Total swap = 0kB
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314891] 985072 pages RAM
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314894] 24998 pages reserved
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314895] 3810 pages shared
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314896] 953288 pages non-shared
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314898] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314906] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314910] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314913] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314916] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314920] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314923] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314926] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314930] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314934] [ 790] 101 790 63477 649 1 0 0 rsyslogd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314937] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314940] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314943] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314946] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314949] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314952] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314956] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314959] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314962] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314965] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314968] [ 654] 1012 654 70351 5781 0 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314971] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314974] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314978] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314981] [13468] 0 13468 89584 2505 0 0 0 php5-fpm
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314984] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314987] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314990] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314994] [19318] 108 19318 6792 75 0 0 0 pickup
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.314997] [27246] 0 27246 8460 79 0 0 0 cron
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315000] [27250] 0 27250 1099 25 1 0 0 sh
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315003] [27251] 0 27251 1074 26 0 0 0 run-parts
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315006] [27254] 0 27254 1099 36 0 0 0 apt
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315009] [27286] 0 27286 1469 23 0 0 0 sleep
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315012] [30119] 1012 30119 828194 764930 0 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315015] [31055] 1012 31055 68580 5604 1 0 0 php
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315018] Out of memory: Kill process 30119 (php) score 798 or sacrifice child
Jul 16 06:29:58 ip-10-61-174-46 kernel: [1483094.315028] Killed process 30119 (php) total-vm:3312776kB, anon-rss:3059720kB, file-rss:0kB
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616128] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616132] php cpuset=/ mems_allowed=0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616135] Pid: 31332, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616137] Call Trace:
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616144] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616146] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616148] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616151] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616155] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616158] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616160] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616164] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616166] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616169] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616171] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616174] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616177] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616180] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616183] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616184] Mem-Info:
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616186] Node 0 DMA per-cpu:
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616187] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616189] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616190] Node 0 DMA32 per-cpu:
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616192] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616193] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616195] active_anon:932532 inactive_anon:1393 isolated_anon:0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616196] active_file:0 inactive_file:0 isolated_file:0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616197] unevictable:0 dirty:22 writeback:0 unstable:0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616197] free:4048 slab_reclaimable:2536 slab_unreclaimable:2653
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616198] mapped:1337 shmem:1405 pagetables:3441 bounce:0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616200] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616206] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616209] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730128kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:88kB writeback:0kB mapped:5348kB shmem:5620kB slab_reclaimable:10144kB slab_unreclaimable:10612kB kernel_stack:816kB pagetables:13764kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616216] lowmem_reserve[]: 0 0 0 0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616218] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616225] Node 0 DMA32: 893*4kB 257*8kB 13*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616231] 1419 total pagecache pages
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616233] 0 pages in swap cache
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616234] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616235] Free swap = 0kB
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.616236] Total swap = 0kB
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621762] 985072 pages RAM
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621765] 24998 pages reserved
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621766] 3814 pages shared
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621767] 953285 pages non-shared
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621769] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621778] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621781] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621784] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621787] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621790] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621794] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621797] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621800] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621804] [ 790] 101 790 63477 649 1 0 0 rsyslogd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621807] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621811] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621814] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621817] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621820] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621823] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621826] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621829] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621832] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621835] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621838] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621842] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621845] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621848] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621851] [13468] 0 13468 89584 2490 1 0 0 php5-fpm
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621854] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621857] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621860] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621864] [19318] 108 19318 6792 75 1 0 0 pickup
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621867] [27246] 0 27246 8460 79 0 0 0 cron
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621870] [27250] 0 27250 1099 25 1 0 0 sh
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621873] [27251] 0 27251 1074 26 0 0 0 run-parts
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621876] [27254] 0 27254 1099 36 0 0 0 apt
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621879] [27286] 0 27286 1469 23 0 0 0 sleep
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621882] [31332] 1012 31332 815266 752038 0 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621885] [ 2249] 108 2249 6795 71 1 0 0 trivial-rewrite
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621888] [ 2250] 108 2250 11107 158 1 0 0 smtp
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621891] [ 2251] 108 2251 11107 158 1 0 0 smtp
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621894] [ 2252] 108 2252 11107 158 1 0 0 smtp
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621897] [ 2949] 1012 2949 68580 5605 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621900] [ 3119] 1012 3119 1099 25 1 0 0 sh
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621903] [ 3120] 1012 3120 68825 5874 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621906] [ 3126] 1012 3126 1099 25 1 0 0 sh
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621909] [ 3127] 1012 3127 14040 63 1 0 0 git
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621912] [ 3128] 1012 3128 68499 5552 1 0 0 php
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621915] [ 3130] 1012 3130 1099 24 1 0 0 sh
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621918] [ 3131] 1012 3131 10901 174 1 0 0 ssh
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621921] Out of memory: Kill process 31332 (php) score 784 or sacrifice child
Jul 16 06:35:32 ip-10-61-174-46 kernel: [1483427.621931] Killed process 31332 (php) total-vm:3261064kB, anon-rss:3008144kB, file-rss:8kB
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812163] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812168] ntpd cpuset=/ mems_allowed=0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812171] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812173] Call Trace:
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812180] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812183] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812186] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812190] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812196] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812200] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812204] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812207] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812210] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812214] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812218] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812221] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812224] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812228] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812233] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812237] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812239] Mem-Info:
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812241] Node 0 DMA per-cpu:
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812243] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812245] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812246] Node 0 DMA32 per-cpu:
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812248] CPU 0: hi: 186, btch: 31 usd: 4
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812250] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812254] active_anon:932638 inactive_anon:1393 isolated_anon:0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812255] active_file:0 inactive_file:17 isolated_file:0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812255] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812256] free:4048 slab_reclaimable:2812 slab_unreclaimable:2635
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812257] mapped:1350 shmem:1405 pagetables:3127 bounce:0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812259] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812268] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812271] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730552kB inactive_anon:5572kB active_file:0kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:11248kB slab_unreclaimable:10540kB kernel_stack:768kB pagetables:12508kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:697 all_unreclaimable? yes
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812288] lowmem_reserve[]: 0 0 0 0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812291] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812301] Node 0 DMA32: 1419*4kB 16*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7900kB
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812310] 1456 total pagecache pages
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812312] 0 pages in swap cache
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812314] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812315] Free swap = 0kB
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.812316] Total swap = 0kB
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817906] 985072 pages RAM
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817909] 24998 pages reserved
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817910] 3813 pages shared
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817912] 953307 pages non-shared
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817913] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817922] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817925] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817928] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817932] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817935] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817938] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817941] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817945] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817949] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817952] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817955] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817959] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817962] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817965] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817968] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817971] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817974] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817977] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817980] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817983] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817986] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817989] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817992] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817995] [13468] 0 13468 89584 2528 1 0 0 php5-fpm
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.817998] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818002] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818005] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818008] [19318] 108 19318 6792 75 0 0 0 pickup
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818011] [27246] 0 27246 8460 79 0 0 0 cron
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818014] [27250] 0 27250 1099 25 1 0 0 sh
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818017] [27251] 0 27251 1074 26 0 0 0 run-parts
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818020] [27254] 0 27254 1099 36 0 0 0 apt
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818023] [27286] 0 27286 1469 23 0 0 0 sleep
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818027] [ 3303] 1012 3303 827732 764403 1 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818030] [12782] 1012 12782 68580 5605 1 0 0 php
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818033] Out of memory: Kill process 3303 (php) score 797 or sacrifice child
Jul 16 06:47:50 ip-10-61-174-46 kernel: [1484165.818044] Killed process 3303 (php) total-vm:3310928kB, anon-rss:3057612kB, file-rss:0kB
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805313] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805317] php cpuset=/ mems_allowed=0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805320] Pid: 13104, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805321] Call Trace:
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805328] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805331] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805333] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805336] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805339] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805343] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805345] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805348] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805351] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805353] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805357] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805359] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805361] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805364] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805366] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805370] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805372] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805374] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805377] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805380] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805381] Mem-Info:
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805382] Node 0 DMA per-cpu:
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805384] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805386] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805387] Node 0 DMA32 per-cpu:
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805388] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805390] CPU 1: hi: 186, btch: 31 usd: 20
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805392] active_anon:932436 inactive_anon:1393 isolated_anon:0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805393] active_file:1 inactive_file:7 isolated_file:0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805393] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805394] free:4025 slab_reclaimable:2845 slab_unreclaimable:2652
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805395] mapped:1353 shmem:1405 pagetables:3224 bounce:0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805396] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805403] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805406] Node 0 DMA32 free:7764kB min:7848kB low:9808kB high:11772kB active_anon:3729744kB inactive_anon:5572kB active_file:4kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5620kB slab_reclaimable:11380kB slab_unreclaimable:10608kB kernel_stack:800kB pagetables:12896kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:95 all_unreclaimable? yes
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805412] lowmem_reserve[]: 0 0 0 0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805415] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805421] Node 0 DMA32: 1263*4kB 81*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7764kB
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805428] 1439 total pagecache pages
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805429] 0 pages in swap cache
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805430] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805432] Free swap = 0kB
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.805432] Total swap = 0kB
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810891] 985072 pages RAM
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810893] 24998 pages reserved
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810895] 3809 pages shared
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810896] 953292 pages non-shared
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810898] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810906] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810910] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810913] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810916] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810919] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810923] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810926] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810929] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810934] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810937] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810940] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810943] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810946] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810950] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810953] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810956] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810959] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810962] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810965] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810968] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810971] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810974] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810977] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810981] [13468] 0 13468 89584 2513 1 0 0 php5-fpm
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810984] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810987] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810990] [ 2686] 1012 2686 68618 5677 1 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810993] [19318] 108 19318 6792 75 1 0 0 pickup
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810996] [27246] 0 27246 8460 79 0 0 0 cron
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.810999] [27250] 0 27250 1099 25 1 0 0 sh
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811002] [27251] 0 27251 1074 26 0 0 0 run-parts
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811005] [27254] 0 27254 1099 36 0 0 0 apt
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811008] [27286] 0 27286 1469 23 0 0 0 sleep
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811012] [13104] 1012 13104 827025 763725 0 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811015] [13679] 108 13679 6818 74 1 0 0 cleanup
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811018] [13680] 108 13680 6795 70 0 0 0 trivial-rewrite
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811021] [13681] 108 13681 11107 158 0 0 0 smtp
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811024] [13733] 108 13733 11107 158 1 0 0 smtp
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811027] [13949] 1012 13949 68580 5604 1 0 0 php
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811030] Out of memory: Kill process 13104 (php) score 797 or sacrifice child
Jul 16 06:49:08 ip-10-61-174-46 kernel: [1484243.811042] Killed process 13104 (php) total-vm:3308100kB, anon-rss:3054900kB, file-rss:0kB
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040085] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040089] php cpuset=/ mems_allowed=0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040092] Pid: 14072, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040093] Call Trace:
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040100] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040103] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040105] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040108] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040113] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040116] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040119] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040122] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040125] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040127] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040130] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040133] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040135] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040138] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040140] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040142] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040145] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040147] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040149] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040152] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040153] Mem-Info:
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040154] Node 0 DMA per-cpu:
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040156] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040157] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040159] Node 0 DMA32 per-cpu:
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040160] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040162] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040164] active_anon:932671 inactive_anon:1393 isolated_anon:0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040165] active_file:0 inactive_file:3 isolated_file:0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040165] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040166] free:4048 slab_reclaimable:2910 slab_unreclaimable:2628
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040167] mapped:1360 shmem:1405 pagetables:3074 bounce:0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040168] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040175] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040178] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730684kB inactive_anon:5572kB active_file:0kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5620kB slab_reclaimable:11640kB slab_unreclaimable:10512kB kernel_stack:728kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:6539 all_unreclaimable? yes
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040184] lowmem_reserve[]: 0 0 0 0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040186] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040193] Node 0 DMA32: 1476*4kB 8*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040200] 1454 total pagecache pages
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040201] 0 pages in swap cache
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040202] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040203] Free swap = 0kB
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.040204] Total swap = 0kB
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045751] 985072 pages RAM
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045753] 24998 pages reserved
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045755] 3794 pages shared
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045756] 953303 pages non-shared
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045758] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045767] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045770] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045774] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045777] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045780] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045783] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045786] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045790] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045794] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045797] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045800] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045804] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045806] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045810] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045813] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045816] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045819] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045822] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045825] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045828] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045831] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045834] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045837] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045840] [13468] 0 13468 89584 2498 1 0 0 php5-fpm
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045844] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045846] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045850] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045853] [19318] 108 19318 6792 75 1 0 0 pickup
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045856] [14072] 1012 14072 827874 764619 0 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045859] [20403] 1012 20403 68580 5604 1 0 0 php
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045862] Out of memory: Kill process 14072 (php) score 798 or sacrifice child
Jul 16 06:57:32 ip-10-61-174-46 kernel: [1484748.045873] Killed process 14072 (php) total-vm:3311496kB, anon-rss:3058476kB, file-rss:0kB
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056084] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056100] php cpuset=/ mems_allowed=0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056103] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056105] Call Trace:
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056113] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056117] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056120] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056124] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056128] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056132] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056135] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056139] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056142] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056146] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056149] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056153] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056157] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056161] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056165] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056168] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056172] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056174] Mem-Info:
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056176] Node 0 DMA per-cpu:
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056178] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056180] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056182] Node 0 DMA32 per-cpu:
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056184] CPU 0: hi: 186, btch: 31 usd: 81
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056186] CPU 1: hi: 186, btch: 31 usd: 26
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056190] active_anon:932680 inactive_anon:1393 isolated_anon:0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056191] active_file:7 inactive_file:19 isolated_file:18
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056192] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056193] free:4046 slab_reclaimable:2469 slab_unreclaimable:2635
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056193] mapped:1344 shmem:1405 pagetables:3312 bounce:0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056196] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056204] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056208] Node 0 DMA32 free:7848kB min:7848kB low:9808kB high:11772kB active_anon:3730720kB inactive_anon:5572kB active_file:28kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):72kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5376kB shmem:5620kB slab_reclaimable:9876kB slab_unreclaimable:10540kB kernel_stack:784kB pagetables:13248kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:227 all_unreclaimable? yes
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056216] lowmem_reserve[]: 0 0 0 0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056220] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056230] Node 0 DMA32: 924*4kB 178*8kB 38*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7936kB
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056240] 1465 total pagecache pages
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056241] 0 pages in swap cache
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056243] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056245] Free swap = 0kB
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.056246] Total swap = 0kB
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061790] 985072 pages RAM
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061791] 24998 pages reserved
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061792] 3789 pages shared
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061793] 953194 pages non-shared
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061795] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061802] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061805] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061807] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061809] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061812] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061814] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061816] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061819] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061822] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061824] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061827] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061829] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061831] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061833] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061836] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061838] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061840] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061842] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061845] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061847] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061849] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061851] [ 658] 1012 658 199421 134839 0 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061853] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061855] [13468] 0 13468 89584 2525 1 0 0 php5-fpm
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061857] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061859] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061862] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061864] [19318] 108 19318 6792 75 0 0 0 pickup
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061867] [20673] 1012 20673 816162 752895 1 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061869] [23523] 1012 23523 68580 5605 1 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061871] [23706] 1012 23706 1099 25 1 0 0 sh
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061873] [23707] 1012 23707 68825 5869 1 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061876] [23713] 1012 23713 1099 26 1 0 0 sh
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061878] [23714] 1012 23714 4655 57 1 0 0 git
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061880] [23715] 1012 23715 68499 5552 1 0 0 php
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061882] [23717] 1012 23717 1099 25 1 0 0 sh
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061884] [23718] 1012 23718 10901 175 1 0 0 ssh
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061886] Out of memory: Kill process 20673 (php) score 785 or sacrifice child
Jul 16 07:01:29 ip-10-61-174-46 kernel: [1484985.061895] Killed process 20673 (php) total-vm:3264648kB, anon-rss:3011580kB, file-rss:0kB
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904138] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904143] php cpuset=/ mems_allowed=0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904146] Pid: 654, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904148] Call Trace:
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904156] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904172] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904175] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904187] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904191] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904195] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904199] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904202] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904205] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904209] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904212] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904215] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904218] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904221] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904224] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904226] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904228] Mem-Info:
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904229] Node 0 DMA per-cpu:
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904231] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904232] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904234] Node 0 DMA32 per-cpu:
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904235] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904236] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904239] active_anon:932984 inactive_anon:1393 isolated_anon:0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904240] active_file:0 inactive_file:0 isolated_file:0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904240] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904241] free:4044 slab_reclaimable:2664 slab_unreclaimable:2593
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904242] mapped:1365 shmem:1405 pagetables:3059 bounce:0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904250] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904253] Node 0 DMA32 free:7840kB min:7848kB low:9808kB high:11772kB active_anon:3731936kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5620kB slab_reclaimable:10656kB slab_unreclaimable:10372kB kernel_stack:720kB pagetables:12236kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904259] lowmem_reserve[]: 0 0 0 0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904262] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904269] Node 0 DMA32: 1428*4kB 14*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7888kB
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904275] 1493 total pagecache pages
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904276] 0 pages in swap cache
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904278] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904279] Free swap = 0kB
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.904280] Total swap = 0kB
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909738] 985072 pages RAM
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909741] 24998 pages reserved
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909742] 3787 pages shared
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909743] 953343 pages non-shared
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909745] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909754] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909757] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909761] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909764] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909767] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909770] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909773] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909777] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909781] [ 790] 101 790 63477 649 1 0 0 rsyslogd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909784] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909787] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909790] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909793] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909796] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909800] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909803] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909806] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909809] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909812] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909815] [ 654] 1012 654 70351 5780 0 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909818] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909821] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909824] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909828] [13468] 0 13468 89584 2510 1 0 0 php5-fpm
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909831] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909834] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909837] [ 2686] 1012 2686 68618 5679 1 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909840] [23801] 1012 23801 828184 765006 0 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909844] [ 6912] 1012 6912 68580 5604 1 0 0 php
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909847] Out of memory: Kill process 23801 (php) score 798 or sacrifice child
Jul 16 07:21:33 ip-10-61-174-46 kernel: [1486188.909858] Killed process 23801 (php) total-vm:3312736kB, anon-rss:3060024kB, file-rss:0kB
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729027] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729030] php cpuset=/ mems_allowed=0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729033] Pid: 7141, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729035] Call Trace:
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729042] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729044] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729046] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729049] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729052] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729056] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729058] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729061] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729064] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729067] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729069] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729072] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729075] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729077] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729080] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729082] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729084] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729087] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729090] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729092] Mem-Info:
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729093] Node 0 DMA per-cpu:
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729095] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729096] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729097] Node 0 DMA32 per-cpu:
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729099] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729100] CPU 1: hi: 186, btch: 31 usd: 27
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729103] active_anon:933313 inactive_anon:1393 isolated_anon:0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729104] active_file:0 inactive_file:6 isolated_file:0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729104] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729105] free:4031 slab_reclaimable:2273 slab_unreclaimable:2604
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729106] mapped:1389 shmem:1405 pagetables:3082 bounce:0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729107] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729114] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729116] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3733252kB inactive_anon:5572kB active_file:0kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5556kB shmem:5620kB slab_reclaimable:9092kB slab_unreclaimable:10416kB kernel_stack:728kB pagetables:12328kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:65 all_unreclaimable? yes
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729123] lowmem_reserve[]: 0 0 0 0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729126] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729132] Node 0 DMA32: 1162*4kB 144*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7880kB
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729139] 1408 total pagecache pages
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729140] 0 pages in swap cache
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729141] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729142] Free swap = 0kB
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.729143] Total swap = 0kB
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734623] 985072 pages RAM
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734625] 24998 pages reserved
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734626] 3780 pages shared
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734627] 953294 pages non-shared
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734629] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734638] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734642] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734645] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734648] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734651] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734655] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734658] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734661] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734665] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734668] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734671] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734675] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734678] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734681] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734684] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734687] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734690] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734693] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734697] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734700] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734703] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734706] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734709] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734712] [13468] 0 13468 89584 2495 1 0 0 php5-fpm
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734715] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734718] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734722] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734725] [ 7141] 1012 7141 828540 765276 0 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734728] [ 7739] 108 7739 6792 70 0 0 0 pickup
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734731] [ 9271] 1012 9271 68580 5605 1 0 0 php
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734734] Out of memory: Kill process 7141 (php) score 798 or sacrifice child
Jul 16 07:24:44 ip-10-61-174-46 kernel: [1486379.734745] Killed process 7141 (php) total-vm:3314160kB, anon-rss:3061104kB, file-rss:0kB
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816487] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816490] php cpuset=/ mems_allowed=0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816493] Pid: 2686, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816494] Call Trace:
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816501] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816504] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816506] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816509] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816514] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816517] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816520] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816523] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816525] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816528] [<ffffffff81192e3d>] ? file_update_time+0xed/0x150
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816531] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816534] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816536] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816539] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816541] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816544] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816547] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816550] [<ffffffff811798c0>] ? vfs_write+0x110/0x180
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816574] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816582] Mem-Info:
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816584] Node 0 DMA per-cpu:
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816586] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816588] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816590] Node 0 DMA32 per-cpu:
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816592] CPU 0: hi: 186, btch: 31 usd: 10
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816594] CPU 1: hi: 186, btch: 31 usd: 36
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816597] active_anon:932818 inactive_anon:1393 isolated_anon:0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816598] active_file:42 inactive_file:20 isolated_file:0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816599] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816600] free:4059 slab_reclaimable:2516 slab_unreclaimable:2584
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816601] mapped:1377 shmem:1405 pagetables:3285 bounce:0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816603] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816611] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816616] Node 0 DMA32 free:7900kB min:7848kB low:9808kB high:11772kB active_anon:3731272kB inactive_anon:5572kB active_file:168kB inactive_file:80kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5508kB shmem:5620kB slab_reclaimable:10064kB slab_unreclaimable:10336kB kernel_stack:768kB pagetables:13140kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2853 all_unreclaimable? yes
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816624] lowmem_reserve[]: 0 0 0 0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816627] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816637] Node 0 DMA32: 813*4kB 239*8kB 34*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816646] 1430 total pagecache pages
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816647] 0 pages in swap cache
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816649] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816650] Free swap = 0kB
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.816652] Total swap = 0kB
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822233] 985072 pages RAM
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822236] 24998 pages reserved
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822237] 3780 pages shared
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822239] 953332 pages non-shared
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822241] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822249] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822253] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822256] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822259] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822262] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822265] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822269] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822272] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822276] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822279] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822282] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822285] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822288] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822291] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822295] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822298] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822301] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822304] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822307] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822310] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822313] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822316] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822319] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822323] [13468] 0 13468 89584 2536 0 0 0 php5-fpm
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822326] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822329] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822332] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822335] [ 7739] 108 7739 6792 70 1 0 0 pickup
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822339] [ 9520] 1012 9520 819618 756348 1 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822341] [14322] 1012 14322 68580 5603 0 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822344] [14466] 1012 14466 1099 25 0 0 0 sh
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822347] [14467] 1012 14467 68825 5873 1 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822350] [14473] 1012 14473 1099 26 0 0 0 sh
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822353] [14474] 1012 14474 4776 57 0 0 0 git
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822356] [14475] 1012 14475 64859 2438 1 0 0 php
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822359] Out of memory: Kill process 9520 (php) score 789 or sacrifice child
Jul 16 07:31:29 ip-10-61-174-46 kernel: [1486784.822370] Killed process 9520 (php) total-vm:3278472kB, anon-rss:3025392kB, file-rss:0kB
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214759] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214762] php cpuset=/ mems_allowed=0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214765] Pid: 678, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214767] Call Trace:
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214774] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214776] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214778] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214781] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214784] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214788] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214790] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214793] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214795] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214799] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214801] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214804] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214807] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214810] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214814] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214816] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214819] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214821] Mem-Info:
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214822] Node 0 DMA per-cpu:
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214824] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214825] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214826] Node 0 DMA32 per-cpu:
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214828] CPU 0: hi: 186, btch: 31 usd: 95
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214829] CPU 1: hi: 186, btch: 31 usd: 93
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214832] active_anon:932672 inactive_anon:1393 isolated_anon:0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214832] active_file:0 inactive_file:2 isolated_file:33
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214833] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214834] free:4018 slab_reclaimable:2696 slab_unreclaimable:2587
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214834] mapped:1355 shmem:1405 pagetables:3157 bounce:0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214836] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214843] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214846] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3730688kB inactive_anon:5572kB active_file:0kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):132kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10784kB slab_unreclaimable:10348kB kernel_stack:744kB pagetables:12628kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:274 all_unreclaimable? yes
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214852] lowmem_reserve[]: 0 0 0 0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214854] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214861] Node 0 DMA32: 1142*4kB 140*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214868] 1470 total pagecache pages
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214869] 0 pages in swap cache
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214870] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214871] Free swap = 0kB
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.214872] Total swap = 0kB
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220366] 985072 pages RAM
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220369] 24998 pages reserved
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220370] 3821 pages shared
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220371] 953171 pages non-shared
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220373] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220382] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220386] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220389] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220392] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220395] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220399] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220402] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220406] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220410] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220413] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220416] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220419] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220422] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220425] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220428] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220431] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220435] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220438] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220441] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220444] [ 654] 1012 654 70351 5783 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220447] [ 657] 1012 657 69242 6260 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220450] [ 658] 1012 658 199421 134842 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220453] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220456] [13468] 0 13468 89584 2521 1 0 0 php5-fpm
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220459] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220462] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220466] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220470] [ 7739] 108 7739 6792 74 1 0 0 pickup
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220473] [15244] 1012 15244 822870 759604 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220476] [17006] 1012 17006 68580 5607 1 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220480] [17160] 1012 17160 1099 25 1 0 0 sh
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220482] [17161] 1012 17161 67881 4978 0 0 0 php
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220485] Out of memory: Kill process 15244 (php) score 792 or sacrifice child
Jul 16 07:35:21 ip-10-61-174-46 kernel: [1487016.220497] Killed process 15244 (php) total-vm:3291480kB, anon-rss:3038404kB, file-rss:12kB
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454087] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454091] php cpuset=/ mems_allowed=0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454094] Pid: 17209, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454096] Call Trace:
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454103] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454105] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454107] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454110] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454113] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454117] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454119] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454122] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454125] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454128] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454131] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454133] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454135] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454138] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454140] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454143] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454146] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454148] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454150] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454153] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454155] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454157] Mem-Info:
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454158] Node 0 DMA per-cpu:
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454160] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454161] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454162] Node 0 DMA32 per-cpu:
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454164] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454165] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454168] active_anon:933016 inactive_anon:1393 isolated_anon:0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454168] active_file:0 inactive_file:20 isolated_file:0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454169] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454170] free:4048 slab_reclaimable:2614 slab_unreclaimable:2610
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454170] mapped:1352 shmem:1405 pagetables:3075 bounce:0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454172] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454179] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454181] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732064kB inactive_anon:5572kB active_file:0kB inactive_file:80kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:10456kB slab_unreclaimable:10440kB kernel_stack:752kB pagetables:12300kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:120 all_unreclaimable? yes
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454188] lowmem_reserve[]: 0 0 0 0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454190] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454198] Node 0 DMA32: 942*4kB 242*8kB 11*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7928kB
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454204] 1451 total pagecache pages
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454205] 0 pages in swap cache
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454207] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454208] Free swap = 0kB
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.454209] Total swap = 0kB
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459597] 985072 pages RAM
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459599] 24998 pages reserved
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459600] 3798 pages shared
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459601] 953320 pages non-shared
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459603] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459612] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459615] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459619] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459621] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459625] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459628] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459631] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459635] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459638] [ 790] 101 790 63477 649 0 0 0 rsyslogd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459642] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459645] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459648] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459651] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459654] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459657] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459660] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459664] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459667] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459670] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459673] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459676] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459680] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459683] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459686] [13468] 0 13468 89584 2506 1 0 0 php5-fpm
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459689] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459692] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459695] [ 2686] 1012 2686 68618 5678 0 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459698] [ 7739] 108 7739 6792 76 1 0 0 pickup
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459702] [17209] 1012 17209 828055 764870 0 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459704] [17967] 108 17967 6792 69 0 0 0 scache
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459708] [20052] 1012 20052 68580 5606 1 0 0 php
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459711] Out of memory: Kill process 17209 (php) score 798 or sacrifice child
Jul 16 07:39:48 ip-10-61-174-46 kernel: [1487284.459722] Killed process 17209 (php) total-vm:3312220kB, anon-rss:3059396kB, file-rss:84kB
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804950] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804954] php cpuset=/ mems_allowed=0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804957] Pid: 29066, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804958] Call Trace:
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804964] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804967] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804969] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804972] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804977] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804980] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804983] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804985] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804988] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804992] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804995] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.804997] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805000] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805003] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805005] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805008] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805010] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805013] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805015] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805017] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805019] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805021] Mem-Info:
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805022] Node 0 DMA per-cpu:
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805024] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805025] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805026] Node 0 DMA32 per-cpu:
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805028] CPU 0: hi: 186, btch: 31 usd: 32
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805030] CPU 1: hi: 186, btch: 31 usd: 60
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805032] active_anon:932878 inactive_anon:1393 isolated_anon:0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805033] active_file:23 inactive_file:4 isolated_file:0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805033] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805034] free:4023 slab_reclaimable:2642 slab_unreclaimable:2607
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805035] mapped:1357 shmem:1405 pagetables:3074 bounce:0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805036] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805043] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805046] Node 0 DMA32 free:7756kB min:7848kB low:9808kB high:11772kB active_anon:3731512kB inactive_anon:5572kB active_file:92kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:10568kB slab_unreclaimable:10428kB kernel_stack:728kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:242 all_unreclaimable? yes
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805052] lowmem_reserve[]: 0 0 0 0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805055] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805062] Node 0 DMA32: 741*4kB 175*8kB 42*16kB 21*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805068] 1450 total pagecache pages
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805070] 0 pages in swap cache
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805071] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805072] Free swap = 0kB
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.805073] Total swap = 0kB
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810456] 985072 pages RAM
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810459] 24998 pages reserved
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810460] 3774 pages shared
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810461] 953296 pages non-shared
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810463] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810472] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810475] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810479] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810482] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810485] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810488] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810491] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810495] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810499] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810502] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810505] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810508] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810511] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810515] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810518] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810521] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810524] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810527] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810530] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810533] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810537] [ 657] 1012 657 69242 6257 1 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810540] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810543] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810546] [13468] 0 13468 89584 2509 0 0 0 php5-fpm
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810549] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810552] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810555] [ 2686] 1012 2686 68618 5679 1 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810559] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810562] [20254] 1012 20254 828066 764803 0 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810565] [29066] 1012 29066 68580 5603 1 0 0 php
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810568] Out of memory: Kill process 20254 (php) score 798 or sacrifice child
Jul 16 07:52:38 ip-10-61-174-46 kernel: [1488053.810579] Killed process 20254 (php) total-vm:3312264kB, anon-rss:3059212kB, file-rss:0kB
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164134] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164139] ntpd cpuset=/ mems_allowed=0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164142] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164144] Call Trace:
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164152] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164155] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164158] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164162] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164171] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164179] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164183] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164187] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164190] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164193] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164197] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164205] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164207] Mem-Info:
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164209] Node 0 DMA per-cpu:
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164211] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164213] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164214] Node 0 DMA32 per-cpu:
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164217] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164222] active_anon:932514 inactive_anon:1393 isolated_anon:0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164223] active_file:0 inactive_file:18 isolated_file:0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164224] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164225] free:4048 slab_reclaimable:2515 slab_unreclaimable:2625
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164226] mapped:1355 shmem:1405 pagetables:3431 bounce:0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164237] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164241] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3730056kB inactive_anon:5572kB active_file:0kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5620kB slab_reclaimable:10060kB slab_unreclaimable:10500kB kernel_stack:824kB pagetables:13724kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:31208 all_unreclaimable? yes
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164249] lowmem_reserve[]: 0 0 0 0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164263] Node 0 DMA32: 710*4kB 234*8kB 40*16kB 12*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7976kB
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164272] 1436 total pagecache pages
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164274] 0 pages in swap cache
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164275] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164276] Free swap = 0kB
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.164277] Total swap = 0kB
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169736] 985072 pages RAM
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169739] 24998 pages reserved
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169740] 3789 pages shared
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169741] 953306 pages non-shared
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169743] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169752] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169755] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169759] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169762] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169765] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169768] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169772] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169775] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169779] [ 790] 101 790 63477 650 0 0 0 rsyslogd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169782] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169785] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169789] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169792] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169795] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169798] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169801] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169804] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169807] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169810] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169813] [ 654] 1012 654 70351 5780 0 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169817] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169820] [ 658] 1012 658 199421 134839 0 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169823] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169826] [13468] 0 13468 89584 2530 0 0 0 php5-fpm
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169829] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169832] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169835] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169839] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169842] [29323] 1012 29323 809313 746081 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169845] [29924] 1012 29924 68580 5603 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169848] [30046] 1012 30046 1099 25 1 0 0 sh
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169851] [30047] 1012 30047 68982 6013 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169854] [30048] 1012 30048 1099 24 1 0 0 sh
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169857] [30049] 1012 30049 69046 6080 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169860] [30050] 1012 30050 1099 25 1 0 0 sh
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169863] [30051] 1012 30051 68982 6015 1 0 0 php
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169866] [30122] 1012 30122 1099 25 1 0 0 sh
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169869] [30123] 1012 30123 1099 25 1 0 0 sh
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169872] [30124] 1012 30124 1099 25 1 0 0 sh
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169875] [30125] 1012 30125 4647 42 1 0 0 git
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169878] [30126] 1012 30126 4647 42 1 0 0 git
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169881] [30127] 1012 30127 4647 42 0 0 0 git
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169884] Out of memory: Kill process 29323 (php) score 778 or sacrifice child
Jul 16 07:53:45 ip-10-61-174-46 kernel: [1488121.169895] Killed process 29323 (php) total-vm:3237252kB, anon-rss:2984324kB, file-rss:0kB
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512223] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512226] php cpuset=/ mems_allowed=0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512229] Pid: 5433, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512231] Call Trace:
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512238] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512240] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512243] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512245] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512250] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512253] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512256] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512258] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512261] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512264] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512267] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512270] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512272] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512275] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512277] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512280] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512282] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512284] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512286] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512288] Mem-Info:
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512289] Node 0 DMA per-cpu:
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512291] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512292] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512293] Node 0 DMA32 per-cpu:
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512295] CPU 0: hi: 186, btch: 31 usd: 6
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512296] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512299] active_anon:932827 inactive_anon:1393 isolated_anon:0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512300] active_file:11 inactive_file:10 isolated_file:0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512300] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512301] free:4048 slab_reclaimable:2663 slab_unreclaimable:2643
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512302] mapped:1347 shmem:1405 pagetables:3179 bounce:0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512303] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512310] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512313] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731308kB inactive_anon:5572kB active_file:44kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5620kB slab_reclaimable:10652kB slab_unreclaimable:10572kB kernel_stack:728kB pagetables:12716kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2626 all_unreclaimable? yes
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512319] lowmem_reserve[]: 0 0 0 0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512322] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512328] Node 0 DMA32: 860*4kB 269*8kB 17*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7912kB
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512335] 1453 total pagecache pages
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512337] 0 pages in swap cache
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512338] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512339] Free swap = 0kB
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.512340] Total swap = 0kB
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517827] 985072 pages RAM
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517830] 24998 pages reserved
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517831] 3786 pages shared
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517832] 953329 pages non-shared
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517834] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517842] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517846] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517849] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517853] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517856] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517859] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517862] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517866] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517870] [ 790] 101 790 63477 650 0 0 0 rsyslogd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517873] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517876] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517880] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517883] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517886] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517889] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517892] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517895] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517898] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517902] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517905] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517908] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517911] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517914] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517917] [13468] 0 13468 89584 2515 1 0 0 php5-fpm
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517920] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517923] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517926] [ 2686] 1012 2686 68618 5679 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517930] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517933] [30185] 1012 30185 823973 760707 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517936] [ 5306] 1012 5306 68580 5604 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517939] [ 5432] 1012 5432 1099 24 1 0 0 sh
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517942] [ 5433] 1012 5433 66483 4057 1 0 0 php
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517945] Out of memory: Kill process 30185 (php) score 793 or sacrifice child
Jul 16 08:05:16 ip-10-61-174-46 kernel: [1488812.517956] Killed process 30185 (php) total-vm:3295892kB, anon-rss:3042828kB, file-rss:0kB
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804645] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804650] collectd cpuset=/ mems_allowed=0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804653] Pid: 1075, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804655] Call Trace:
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804663] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804666] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804669] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804673] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804677] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804681] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804684] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804688] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804693] [<ffffffff8165e56e>] ? _raw_spin_lock_irqsave+0x2e/0x40
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804699] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804702] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804706] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804709] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804712] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804715] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804718] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804723] [<ffffffff810a152c>] ? do_futex+0x7c/0x1b0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804726] [<ffffffff810a17a2>] ? sys_futex+0x142/0x1a0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804729] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804733] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804735] Mem-Info:
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804736] Node 0 DMA per-cpu:
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804739] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804741] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804742] Node 0 DMA32 per-cpu:
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804745] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804746] CPU 1: hi: 186, btch: 31 usd: 1
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804750] active_anon:933350 inactive_anon:1393 isolated_anon:0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804751] active_file:9 inactive_file:38 isolated_file:0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804752] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804752] free:4023 slab_reclaimable:2251 slab_unreclaimable:2570
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804753] mapped:1341 shmem:1405 pagetables:3074 bounce:0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804756] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804764] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804768] Node 0 DMA32 free:7756kB min:7848kB low:9808kB high:11772kB active_anon:3733400kB inactive_anon:5572kB active_file:36kB inactive_file:152kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5364kB shmem:5620kB slab_reclaimable:9004kB slab_unreclaimable:10280kB kernel_stack:728kB pagetables:12296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:118 all_unreclaimable? yes
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804776] lowmem_reserve[]: 0 0 0 0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804780] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804789] Node 0 DMA32: 1023*4kB 202*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804798] 1467 total pagecache pages
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804799] 0 pages in swap cache
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804801] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804803] Free swap = 0kB
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.804804] Total swap = 0kB
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810340] 985072 pages RAM
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810342] 24998 pages reserved
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810343] 3783 pages shared
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810345] 953348 pages non-shared
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810346] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810355] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810359] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810362] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810365] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810369] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810372] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810375] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810379] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810383] [ 790] 101 790 63477 650 0 0 0 rsyslogd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810386] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810389] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810392] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810395] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810398] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810401] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810405] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810408] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810411] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810414] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810418] [ 654] 1012 654 70351 5780 0 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810421] [ 657] 1012 657 69242 6258 0 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810424] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810427] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810430] [13468] 0 13468 89584 2500 0 0 0 php5-fpm
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810433] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810436] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810439] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810443] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810446] [ 5495] 1012 5495 828578 765277 0 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810450] [ 6856] 1012 6856 68580 5604 0 0 0 php
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810452] Out of memory: Kill process 5495 (php) score 798 or sacrifice child
Jul 16 08:07:33 ip-10-61-174-46 kernel: [1488948.810463] Killed process 5495 (php) total-vm:3314312kB, anon-rss:3061108kB, file-rss:0kB
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608123] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608128] php cpuset=/ mems_allowed=0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608131] Pid: 7006, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608134] Call Trace:
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608142] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608145] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608148] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608152] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608158] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608161] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608166] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608169] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608172] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608174] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608178] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608180] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608182] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608185] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608188] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608191] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608193] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608195] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608198] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608201] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608202] Mem-Info:
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608203] Node 0 DMA per-cpu:
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608205] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608207] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608208] Node 0 DMA32 per-cpu:
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608209] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608211] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608213] active_anon:933225 inactive_anon:1393 isolated_anon:0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608214] active_file:0 inactive_file:0 isolated_file:0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608214] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608215] free:4048 slab_reclaimable:2353 slab_unreclaimable:2626
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608216] mapped:1364 shmem:1405 pagetables:3058 bounce:0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608217] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608224] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608227] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732900kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5620kB slab_reclaimable:9412kB slab_unreclaimable:10504kB kernel_stack:728kB pagetables:12232kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608233] lowmem_reserve[]: 0 0 0 0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608236] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608242] Node 0 DMA32: 936*4kB 226*8kB 8*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7856kB
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608249] 1467 total pagecache pages
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608250] 0 pages in swap cache
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608252] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608253] Free swap = 0kB
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.608254] Total swap = 0kB
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613782] 985072 pages RAM
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613784] 24998 pages reserved
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613785] 3815 pages shared
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613786] 953324 pages non-shared
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613788] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613797] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613800] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613804] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613807] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613810] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613813] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613817] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613820] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613824] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613827] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613830] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613833] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613836] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613839] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613843] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613846] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613849] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613852] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613855] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613858] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613861] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613864] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613867] [ 678] 1012 678 70152 5568 1 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613871] [13468] 0 13468 89584 2535 1 0 0 php5-fpm
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613874] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613877] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613880] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613883] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613887] [ 7006] 1012 7006 828450 765199 0 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613890] [ 7664] 1012 7664 68580 5605 1 0 0 php
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613892] Out of memory: Kill process 7006 (php) score 798 or sacrifice child
Jul 16 08:08:44 ip-10-61-174-46 kernel: [1489019.613903] Killed process 7006 (php) total-vm:3313800kB, anon-rss:3060760kB, file-rss:36kB
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016550] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016555] php cpuset=/ mems_allowed=0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016558] Pid: 7811, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016560] Call Trace:
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016568] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016571] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016574] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016578] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016582] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016587] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016590] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016594] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016600] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016603] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016607] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016610] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016614] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016617] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016621] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016624] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016626] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016629] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016633] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016637] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016639] Mem-Info:
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016641] Node 0 DMA per-cpu:
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016643] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016645] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016647] Node 0 DMA32 per-cpu:
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016649] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016651] CPU 1: hi: 186, btch: 31 usd: 66
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016654] active_anon:932935 inactive_anon:1393 isolated_anon:0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016655] active_file:2 inactive_file:0 isolated_file:0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016656] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016657] free:4018 slab_reclaimable:2632 slab_unreclaimable:2593
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016658] mapped:1350 shmem:1405 pagetables:3070 bounce:0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016660] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016668] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016672] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3731740kB inactive_anon:5572kB active_file:8kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5620kB slab_reclaimable:10528kB slab_unreclaimable:10372kB kernel_stack:728kB pagetables:12280kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:150 all_unreclaimable? yes
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016681] lowmem_reserve[]: 0 0 0 0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016684] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016691] Node 0 DMA32: 908*4kB 257*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7736kB
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016698] 1461 total pagecache pages
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016699] 0 pages in swap cache
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016700] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016701] Free swap = 0kB
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.016702] Total swap = 0kB
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022186] 985072 pages RAM
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022189] 24998 pages reserved
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022190] 3785 pages shared
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022191] 953286 pages non-shared
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022193] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022201] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022205] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022208] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022211] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022214] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022218] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022221] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022224] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022229] [ 790] 101 790 63477 650 0 0 0 rsyslogd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022232] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022235] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022238] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022241] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022244] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022247] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022250] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022253] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022257] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022260] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022263] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022266] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022269] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022272] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022275] [13468] 0 13468 89584 2520 1 0 0 php5-fpm
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022278] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022281] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022284] [ 2686] 1012 2686 68618 5679 1 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022288] [ 7739] 108 7739 6792 76 1 0 0 pickup
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022291] [ 7811] 1012 7811 828161 764903 0 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022294] [11957] 1012 11957 68580 5604 1 0 0 php
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022297] Out of memory: Kill process 7811 (php) score 798 or sacrifice child
Jul 16 08:15:06 ip-10-61-174-46 kernel: [1489402.022307] Killed process 7811 (php) total-vm:3312644kB, anon-rss:3059612kB, file-rss:0kB
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904139] php cpuset=/ mems_allowed=0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904142] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904144] Call Trace:
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904152] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904155] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904158] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904162] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904171] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904181] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904185] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904188] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904197] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904201] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904205] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904208] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904212] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904215] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904218] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904220] Mem-Info:
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904222] Node 0 DMA per-cpu:
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904224] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904226] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904228] Node 0 DMA32 per-cpu:
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904230] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904232] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904235] active_anon:932440 inactive_anon:1393 isolated_anon:0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904236] active_file:1 inactive_file:3 isolated_file:0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904237] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904238] free:4048 slab_reclaimable:2465 slab_unreclaimable:2654
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904239] mapped:1357 shmem:1405 pagetables:3559 bounce:0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904241] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904249] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904253] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729760kB inactive_anon:5572kB active_file:4kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9860kB slab_unreclaimable:10616kB kernel_stack:840kB pagetables:14236kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4927 all_unreclaimable? yes
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904261] lowmem_reserve[]: 0 0 0 0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904265] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904274] Node 0 DMA32: 754*4kB 252*8kB 39*16kB 7*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7928kB
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904283] 1441 total pagecache pages
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904285] 0 pages in swap cache
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904287] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904288] Free swap = 0kB
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.904289] Total swap = 0kB
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909470] 985072 pages RAM
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909472] 24998 pages reserved
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909473] 3794 pages shared
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909474] 953324 pages non-shared
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909476] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909485] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909489] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909492] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909495] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909498] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909501] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909505] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909508] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909512] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909515] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909518] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909522] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909525] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909528] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909531] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909534] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909537] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909540] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909543] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909546] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909549] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909553] [ 658] 1012 658 199421 134840 0 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909556] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909559] [13468] 0 13468 89584 2505 1 0 0 php5-fpm
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909562] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909565] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909568] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909572] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909576] [12074] 1012 12074 804129 740862 0 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909579] [15335] 1012 15335 68580 5605 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909582] [15517] 1012 15517 1099 25 0 0 0 sh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909585] [15518] 1012 15518 68825 5868 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909588] [15521] 1012 15521 1099 24 1 0 0 sh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909591] [15522] 1012 15522 68825 5873 1 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909593] [15528] 1012 15528 1099 26 0 0 0 sh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909596] [15529] 1012 15529 4724 58 0 0 0 git
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909599] [15530] 1012 15530 68499 5551 0 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909602] [15535] 1012 15535 1099 25 0 0 0 sh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909605] [15537] 1012 15537 14042 62 0 0 0 git
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909608] [15538] 1012 15538 68499 5551 0 0 0 php
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909611] [15540] 1012 15540 1099 24 0 0 0 sh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909614] [15541] 1012 15541 10901 175 1 0 0 ssh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909617] [15542] 1012 15542 1099 25 1 0 0 sh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909620] [15543] 1012 15543 10901 177 1 0 0 ssh
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909623] Out of memory: Kill process 12074 (php) score 773 or sacrifice child
Jul 16 08:20:09 ip-10-61-174-46 kernel: [1489704.909635] Killed process 12074 (php) total-vm:3216516kB, anon-rss:2963448kB, file-rss:0kB
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584129] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584133] php cpuset=/ mems_allowed=0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584136] Pid: 15580, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584137] Call Trace:
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584145] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584147] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584149] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584152] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584156] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584159] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584162] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584165] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584167] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584170] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584172] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584175] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584178] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584181] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584183] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584186] [<ffffffff8165c2f1>] ? __schedule+0x6b1/0x6f0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584188] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584190] Mem-Info:
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584191] Node 0 DMA per-cpu:
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584193] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584194] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584196] Node 0 DMA32 per-cpu:
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584197] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584198] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584201] active_anon:933061 inactive_anon:1393 isolated_anon:0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584202] active_file:25 inactive_file:37 isolated_file:0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584203] unevictable:0 dirty:19 writeback:0 unstable:0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584203] free:4048 slab_reclaimable:2531 slab_unreclaimable:2585
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584204] mapped:1370 shmem:1405 pagetables:3053 bounce:0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584206] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584212] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584215] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732244kB inactive_anon:5572kB active_file:100kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:76kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:10124kB slab_unreclaimable:10340kB kernel_stack:728kB pagetables:12212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24336 all_unreclaimable? yes
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584222] lowmem_reserve[]: 0 0 0 0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584224] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584231] Node 0 DMA32: 886*4kB 138*8kB 62*16kB 7*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7912kB
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584238] 1453 total pagecache pages
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584239] 0 pages in swap cache
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584240] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584241] Free swap = 0kB
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.584242] Total swap = 0kB
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589949] 985072 pages RAM
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589952] 24998 pages reserved
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589953] 3798 pages shared
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589954] 953325 pages non-shared
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589956] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589964] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589968] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589971] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589974] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589977] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589981] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589984] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589988] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589991] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589995] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.589998] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590001] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590004] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590007] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590010] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590013] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590016] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590020] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590023] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590026] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590029] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590032] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590035] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590038] [13468] 0 13468 89584 2490 1 0 0 php5-fpm
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590041] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590044] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590047] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590051] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590055] [15580] 1012 15580 828258 765035 0 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590058] [16563] 1012 16563 68580 5604 1 0 0 php
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590061] Out of memory: Kill process 15580 (php) score 798 or sacrifice child
Jul 16 08:22:06 ip-10-61-174-46 kernel: [1489821.590071] Killed process 15580 (php) total-vm:3313032kB, anon-rss:3060052kB, file-rss:88kB
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756112] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756116] php cpuset=/ mems_allowed=0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756118] Pid: 16851, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756120] Call Trace:
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756127] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756129] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756131] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756134] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756137] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756141] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756143] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756146] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756150] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756153] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756156] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756158] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756161] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756163] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756165] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756168] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756170] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756173] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756175] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756177] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756180] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756182] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756184] Mem-Info:
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756185] Node 0 DMA per-cpu:
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756187] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756188] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756189] Node 0 DMA32 per-cpu:
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756191] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756192] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756195] active_anon:933039 inactive_anon:1393 isolated_anon:0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756195] active_file:0 inactive_file:0 isolated_file:0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756196] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756197] free:4048 slab_reclaimable:2620 slab_unreclaimable:2572
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756197] mapped:1347 shmem:1405 pagetables:3080 bounce:0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756199] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756205] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756208] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732156kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5620kB slab_reclaimable:10480kB slab_unreclaimable:10288kB kernel_stack:728kB pagetables:12320kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756214] lowmem_reserve[]: 0 0 0 0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756217] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756224] Node 0 DMA32: 1094*4kB 179*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7872kB
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756230] 1432 total pagecache pages
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756232] 0 pages in swap cache
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756233] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756234] Free swap = 0kB
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.756235] Total swap = 0kB
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761573] 985072 pages RAM
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761576] 24998 pages reserved
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761577] 3795 pages shared
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761578] 953332 pages non-shared
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761580] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761589] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761592] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761595] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761599] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761602] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761605] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761608] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761612] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761615] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761619] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761622] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761625] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761628] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761631] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761634] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761638] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761641] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761644] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761647] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761650] [ 654] 1012 654 70351 5780 1 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761653] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761656] [ 658] 1012 658 199421 134839 1 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761659] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761663] [13468] 0 13468 89584 2509 1 0 0 php5-fpm
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761666] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761669] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761672] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761676] [ 7739] 108 7739 6792 76 1 0 0 pickup
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761679] [16851] 1012 16851 828222 765002 0 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761682] [19634] 1012 19634 68580 5605 1 0 0 php
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761684] Out of memory: Kill process 16851 (php) score 798 or sacrifice child
Jul 16 08:26:37 ip-10-61-174-46 kernel: [1490092.761695] Killed process 16851 (php) total-vm:3312888kB, anon-rss:3059984kB, file-rss:24kB
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.115993] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.115997] ntpd cpuset=/ mems_allowed=0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.115999] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116002] Call Trace:
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116008] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116011] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116013] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116016] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116019] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116022] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116024] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116027] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116032] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116034] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116037] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116039] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116042] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116044] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116047] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116049] Mem-Info:
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116064] Node 0 DMA per-cpu:
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116067] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116069] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116070] Node 0 DMA32 per-cpu:
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116072] CPU 0: hi: 186, btch: 31 usd: 26
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116074] CPU 1: hi: 186, btch: 31 usd: 37
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116078] active_anon:932618 inactive_anon:1393 isolated_anon:0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116079] active_file:11 inactive_file:16 isolated_file:19
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116079] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116080] free:4039 slab_reclaimable:2547 slab_unreclaimable:2602
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116081] mapped:1354 shmem:1405 pagetables:3368 bounce:0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116083] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116092] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116096] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3730472kB inactive_anon:5572kB active_file:44kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5620kB slab_reclaimable:10188kB slab_unreclaimable:10408kB kernel_stack:768kB pagetables:13472kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:182 all_unreclaimable? yes
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116104] lowmem_reserve[]: 0 0 0 0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116108] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116118] Node 0 DMA32: 1107*4kB 168*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116128] 1459 total pagecache pages
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116130] 0 pages in swap cache
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116131] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116133] Free swap = 0kB
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.116134] Total swap = 0kB
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121682] 985072 pages RAM
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121684] 24998 pages reserved
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121685] 3776 pages shared
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121686] 953302 pages non-shared
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121688] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121695] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121698] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121700] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121702] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121705] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121707] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121709] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121712] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121715] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121717] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121719] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121722] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121724] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121726] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121728] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121730] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121733] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121735] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121737] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121739] [ 654] 1012 654 70351 5781 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121742] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121744] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121746] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121749] [13468] 0 13468 89584 2519 0 0 0 php5-fpm
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121751] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121753] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121755] [ 2686] 1012 2686 68618 5678 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121758] [ 7739] 108 7739 6792 76 0 0 0 pickup
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121760] [19823] 1012 19823 815869 752604 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121762] [20893] 1012 20893 68580 5603 0 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121765] [21080] 1012 21080 1099 25 1 0 0 sh
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121767] [21081] 1012 21081 1099 24 1 0 0 sh
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121769] [21082] 1012 21082 66418 3983 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121771] [21083] 1012 21083 67296 4330 1 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121773] [21084] 1012 21084 1099 25 0 0 0 sh
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121775] [21085] 1012 21085 66028 3572 0 0 0 php
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121777] Out of memory: Kill process 19823 (php) score 785 or sacrifice child
Jul 16 08:28:32 ip-10-61-174-46 kernel: [1490207.121787] Killed process 19823 (php) total-vm:3263476kB, anon-rss:3010416kB, file-rss:0kB
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016143] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016148] php cpuset=/ mems_allowed=0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016151] Pid: 658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016153] Call Trace:
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016164] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016167] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016170] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016174] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016178] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016183] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016188] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016191] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016195] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016198] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016201] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016205] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016208] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016211] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016216] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016219] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016223] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016226] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016230] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016233] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016237] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016239] Mem-Info:
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016240] Node 0 DMA per-cpu:
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016243] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016245] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016246] Node 0 DMA32 per-cpu:
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016248] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016250] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016254] active_anon:933055 inactive_anon:1393 isolated_anon:0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016255] active_file:5 inactive_file:15 isolated_file:0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016256] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016256] free:4048 slab_reclaimable:2351 slab_unreclaimable:2586
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016257] mapped:1352 shmem:1405 pagetables:3197 bounce:0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016260] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016268] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016272] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3732220kB inactive_anon:5572kB active_file:20kB inactive_file:60kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9404kB slab_unreclaimable:10344kB kernel_stack:760kB pagetables:12788kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:351 all_unreclaimable? yes
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016280] lowmem_reserve[]: 0 0 0 0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016284] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016291] Node 0 DMA32: 991*4kB 232*8kB 7*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8012kB
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016298] 1425 total pagecache pages
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016299] 0 pages in swap cache
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016300] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016301] Free swap = 0kB
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.016302] Total swap = 0kB
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022396] 985072 pages RAM
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022399] 24998 pages reserved
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022400] 3779 pages shared
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022401] 953315 pages non-shared
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022403] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022411] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022415] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022418] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022420] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022423] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022426] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022429] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022432] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022436] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022439] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022442] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022446] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022449] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022452] [ 1024] 0 1024 3309 38 1 0 0 mdadm
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022455] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022458] [ 1051] 0 1051 201628 845 1 0 0 collectd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022460] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022463] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022466] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022469] [ 654] 1012 654 70351 5781 0 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022471] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022474] [ 658] 1012 658 199421 134840 1 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022476] [ 678] 1012 678 70152 5566 0 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022479] [13468] 0 13468 89584 2537 0 0 0 php5-fpm
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022483] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022485] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022488] [ 2686] 1012 2686 68618 5679 1 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022492] [ 7739] 108 7739 6792 76 1 0 0 pickup
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022495] [21173] 1012 21173 822306 759056 0 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022500] [25427] 1012 25427 68580 5603 0 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022503] [25551] 1012 25551 1099 25 0 0 0 sh
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022506] [25552] 1012 25552 68825 5850 0 0 0 php
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022508] [25554] 1012 25554 1099 25 0 0 0 sh
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022511] [25555] 1012 25555 4647 45 1 0 0 git
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022514] Out of memory: Kill process 21173 (php) score 792 or sacrifice child
Jul 16 08:35:35 ip-10-61-174-46 kernel: [1490631.022525] Killed process 21173 (php) total-vm:3289224kB, anon-rss:3036224kB, file-rss:0kB
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408615] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408619] php cpuset=/ mems_allowed=0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408622] Pid: 25671, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408624] Call Trace:
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408631] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408633] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408635] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408638] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408641] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408645] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408647] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408650] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408653] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408656] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408659] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408661] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408664] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408666] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408669] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408672] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408674] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408676] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408679] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408682] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408684] Mem-Info:
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408685] Node 0 DMA per-cpu:
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408687] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408688] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408689] Node 0 DMA32 per-cpu:
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408691] CPU 0: hi: 186, btch: 31 usd: 1
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408692] CPU 1: hi: 186, btch: 31 usd: 32
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408695] active_anon:931987 inactive_anon:1394 isolated_anon:0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408696] active_file:82 inactive_file:78 isolated_file:22
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408696] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408697] free:4048 slab_reclaimable:2644 slab_unreclaimable:2649
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408698] mapped:1371 shmem:1406 pagetables:3538 bounce:0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408699] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408706] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408709] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727948kB inactive_anon:5576kB active_file:328kB inactive_file:312kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5484kB shmem:5624kB slab_reclaimable:10576kB slab_unreclaimable:10596kB kernel_stack:872kB pagetables:14152kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:989 all_unreclaimable? yes
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408715] lowmem_reserve[]: 0 0 0 0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408718] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408725] Node 0 DMA32: 1313*4kB 77*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7948kB
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408731] 1572 total pagecache pages
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408732] 0 pages in swap cache
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408734] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408735] Free swap = 0kB
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.408736] Total swap = 0kB
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414400] 985072 pages RAM
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414402] 24998 pages reserved
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414403] 4186 pages shared
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414405] 952897 pages non-shared
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414407] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414415] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414418] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414422] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414425] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414428] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414431] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414435] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414438] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414442] [ 790] 101 790 63477 650 0 0 0 rsyslogd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414445] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414448] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414452] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414455] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414458] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414461] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414464] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414467] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414471] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414474] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414477] [ 654] 1012 654 70351 5781 0 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414480] [ 657] 1012 657 69242 6258 1 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414483] [ 658] 1012 658 199421 134839 0 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414486] [ 678] 1012 678 70152 5566 1 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414490] [13468] 0 13468 89584 2521 0 0 0 php5-fpm
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414493] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414495] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414499] [ 2686] 1012 2686 68618 5679 1 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414502] [25671] 1012 25671 781088 717852 1 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414506] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414509] [ 1215] 1002 1215 18359 197 0 0 0 sshd
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414512] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414515] [ 1217] 0 1217 2786 86 0 0 0 muppet
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414518] [ 1255] 0 1255 52972 27061 0 0 0 puppet
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414521] [ 1256] 0 1256 1469 25 0 0 0 tee
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414524] [ 1257] 0 1257 2786 85 1 0 0 muppet
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414527] [ 1258] 0 1258 2554 30 0 0 0 sed
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414530] [ 1259] 0 1259 1474 24 1 0 0 tr
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414532] [ 1260] 0 1260 1540 23 1 0 0 xargs
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414535] [ 1767] 0 1767 20773 2575 1 0 0 apt-get
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414538] [ 1833] 1012 1833 68580 5604 1 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414541] [ 1846] 0 1846 9555 5308 0 0 0 dpkg
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414544] [ 1852] 1012 1852 1099 25 0 0 0 sh
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414547] [ 1853] 1012 1853 68206 5266 0 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414551] [ 1854] 1012 1854 1099 25 1 0 0 sh
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414553] [ 1855] 1012 1855 68207 5263 0 0 0 php
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414556] [ 1865] 0 1865 2180 29 0 0 0 dpkg-deb
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414559] [ 1866] 0 1866 2180 34 1 0 0 dpkg-deb
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414562] [ 1867] 0 1867 2181 44 0 0 0 dpkg-deb
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414565] Out of memory: Kill process 25671 (php) score 749 or sacrifice child
Jul 16 08:48:41 ip-10-61-174-46 kernel: [1491417.414577] Killed process 25671 (php) total-vm:3124352kB, anon-rss:2871408kB, file-rss:0kB
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116022] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116027] ntpd cpuset=/ mems_allowed=0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116030] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116032] Call Trace:
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116039] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116042] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116044] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116047] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116052] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116069] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116072] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116076] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116079] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116083] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116086] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116090] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116094] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116098] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116102] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116104] Mem-Info:
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116106] Node 0 DMA per-cpu:
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116108] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116110] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116112] Node 0 DMA32 per-cpu:
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116114] CPU 0: hi: 186, btch: 31 usd: 72
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116116] CPU 1: hi: 186, btch: 31 usd: 49
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116120] active_anon:933010 inactive_anon:1394 isolated_anon:0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116121] active_file:33 inactive_file:0 isolated_file:11
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116122] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116123] free:4045 slab_reclaimable:2569 slab_unreclaimable:2621
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116124] mapped:1379 shmem:1406 pagetables:2905 bounce:0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116126] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116134] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116138] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3732040kB inactive_anon:5576kB active_file:132kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):44kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5624kB slab_reclaimable:10276kB slab_unreclaimable:10484kB kernel_stack:784kB pagetables:11620kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:334 all_unreclaimable? yes
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116147] lowmem_reserve[]: 0 0 0 0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116150] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116162] Node 0 DMA32: 1459*4kB 4*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7916kB
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116172] 1443 total pagecache pages
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116173] 0 pages in swap cache
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116175] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116177] Free swap = 0kB
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.116178] Total swap = 0kB
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121770] 985072 pages RAM
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121772] 24998 pages reserved
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121774] 4015 pages shared
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121775] 952979 pages non-shared
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121777] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121785] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121787] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121790] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121792] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121794] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121797] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121799] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121801] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121804] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121807] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121809] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121811] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121813] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121815] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121818] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121820] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121822] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121824] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121826] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121829] [ 654] 1012 654 70351 5782 1 0 0 php
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121831] [13468] 0 13468 89584 2534 0 0 0 php5-fpm
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121833] [13470] 33 13470 89584 1189 0 0 0 php5-fpm
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121835] [13471] 33 13471 89584 1189 0 0 0 php5-fpm
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121838] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121840] [ 1215] 1002 1215 18359 200 0 0 0 sshd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121843] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121845] [ 1217] 0 1217 2786 86 0 0 0 muppet
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121847] [ 1255] 0 1255 53265 27347 1 0 0 puppet
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121849] [ 1256] 0 1256 1469 25 1 0 0 tee
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121851] [ 1257] 0 1257 2786 85 1 0 0 muppet
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121853] [ 1258] 0 1258 2554 42 1 0 0 sed
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121855] [ 1259] 0 1259 1475 24 1 0 0 tr
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121857] [ 1260] 0 1260 1540 27 0 0 0 xargs
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121859] [ 1833] 1012 1833 952554 889373 0 0 0 php
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121862] [ 3430] 0 3430 1099 25 1 0 0 phd
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121864] [ 3432] 0 3432 68230 5648 0 0 0 php
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121866] Out of memory: Kill process 1833 (php) score 928 or sacrifice child
Jul 16 08:49:14 ip-10-61-174-46 kernel: [1491450.121875] Killed process 1833 (php) total-vm:3810216kB, anon-rss:3557488kB, file-rss:4kB
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344128] frontend invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344132] frontend cpuset=/ mems_allowed=0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344135] Pid: 6222, comm: frontend Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344137] Call Trace:
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344144] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344146] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344148] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344151] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344155] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344158] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344160] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344163] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344168] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344170] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344173] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344176] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344178] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344181] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344183] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344186] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344188] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344192] [<ffffffff8117e6aa>] ? sys_newstat+0x2a/0x40
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344194] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344196] Mem-Info:
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344197] Node 0 DMA per-cpu:
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344199] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344200] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344201] Node 0 DMA32 per-cpu:
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344203] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344204] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344207] active_anon:929919 inactive_anon:1395 isolated_anon:0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344207] active_file:11 inactive_file:4 isolated_file:0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344208] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344209] free:3285 slab_reclaimable:2769 slab_unreclaimable:3230
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344209] mapped:1364 shmem:1406 pagetables:4337 bounce:0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344211] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344217] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344220] Node 0 DMA32 free:4804kB min:7848kB low:9808kB high:11772kB active_anon:3719676kB inactive_anon:5580kB active_file:44kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5456kB shmem:5624kB slab_reclaimable:11076kB slab_unreclaimable:12920kB kernel_stack:1024kB pagetables:17348kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:7784 all_unreclaimable? yes
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344227] lowmem_reserve[]: 0 0 0 0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344229] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344236] Node 0 DMA32: 402*4kB 51*8kB 35*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4816kB
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344243] 1426 total pagecache pages
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344244] 0 pages in swap cache
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344246] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344247] Free swap = 0kB
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.344248] Total swap = 0kB
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349859] 985072 pages RAM
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349862] 24998 pages reserved
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349863] 3992 pages shared
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349864] 953871 pages non-shared
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349866] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349875] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349878] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349881] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349884] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349888] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349891] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349894] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349898] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349902] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349905] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349908] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349911] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349914] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349917] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349921] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349924] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349927] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349930] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349933] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349937] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349940] [ 1215] 1002 1215 18359 200 0 0 0 sshd
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349943] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349946] [ 1217] 0 1217 2786 86 0 0 0 muppet
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349949] [ 1255] 0 1255 53209 27322 1 0 0 puppet
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349952] [ 1256] 0 1256 1469 25 0 0 0 tee
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349955] [ 1257] 0 1257 2786 85 1 0 0 muppet
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349958] [ 1258] 0 1258 2554 42 0 0 0 sed
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349961] [ 1259] 0 1259 1475 24 1 0 0 tr
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349964] [ 1260] 0 1260 1540 27 0 0 0 xargs
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349967] [ 3502] 1012 3502 69878 5698 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349970] [ 3504] 1012 3504 68832 6168 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349973] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349976] [ 3508] 1012 3508 487740 424826 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349980] [ 3526] 1012 3526 69673 5533 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349982] [ 3533] 1012 3533 68271 5618 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349985] [ 3691] 1012 3691 331891 269048 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349988] [ 3764] 0 3764 20774 2577 0 0 0 apt-get
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349991] [ 3867] 1012 3867 156978 93549 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349994] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.349997] [ 4460] 1012 4460 1099 24 1 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350001] [ 4475] 1012 4475 68449 5859 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350004] [ 4693] 0 4693 2908 531 0 0 0 dpkg
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350007] [ 5213] 1012 5213 107642 39451 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350010] [ 5415] 1012 5415 1099 24 0 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350013] [ 5417] 1012 5417 68424 5869 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350016] [ 5518] 1012 5518 1099 25 0 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350019] [ 5519] 1012 5519 68424 5869 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350022] [ 5521] 1012 5521 1099 25 1 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350025] [ 5523] 1012 5523 68424 5872 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350028] [ 6008] 1012 6008 1099 25 0 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350031] [ 6009] 1012 6009 4776 58 0 0 0 git
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350034] [ 6018] 1012 6018 66568 4087 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350036] [ 6125] 1012 6125 1099 25 0 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350040] [ 6136] 1012 6136 4650 53 0 0 0 git
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350043] [ 6150] 1012 6150 65256 2825 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350046] [ 6191] 1012 6191 1099 25 1 0 0 sh
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350049] [ 6203] 1012 6203 4984 62 1 0 0 git
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350051] [ 6211] 1012 6211 67117 4598 0 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350055] [ 6212] 1012 6212 64490 1836 1 0 0 php
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350058] [ 6218] 0 6218 89584 2537 1 0 0 php5-fpm
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350061] [ 6221] 0 6221 1099 26 1 0 0 php5-gd.postins
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350064] [ 6222] 0 6222 7211 819 0 0 0 frontend
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350067] [ 6240] 33 6240 89584 1190 0 0 0 php5-fpm
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350070] [ 6242] 33 6242 89584 1190 1 0 0 php5-fpm
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350072] Out of memory: Kill process 3508 (php) score 443 or sacrifice child
Jul 16 08:49:43 ip-10-61-174-46 kernel: [1491479.350084] Killed process 3508 (php) total-vm:1950960kB, anon-rss:1699304kB, file-rss:0kB
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808158] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808163] php cpuset=/ mems_allowed=0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808166] Pid: 5213, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808168] Call Trace:
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808176] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808179] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808182] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808186] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808190] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808195] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808198] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808203] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808207] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808211] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808215] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808218] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808222] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808225] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808228] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808232] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808236] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808240] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808242] Mem-Info:
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808244] Node 0 DMA per-cpu:
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808246] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808248] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808249] Node 0 DMA32 per-cpu:
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808252] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808253] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808257] active_anon:931252 inactive_anon:1395 isolated_anon:0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808258] active_file:23 inactive_file:7 isolated_file:0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808259] unevictable:0 dirty:5 writeback:6 unstable:0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808260] free:2827 slab_reclaimable:2636 slab_unreclaimable:3151
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808261] mapped:1360 shmem:1406 pagetables:3959 bounce:0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808263] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808271] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808275] Node 0 DMA32 free:2972kB min:7848kB low:9808kB high:11772kB active_anon:3725008kB inactive_anon:5580kB active_file:92kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:20kB writeback:24kB mapped:5440kB shmem:5624kB slab_reclaimable:10544kB slab_unreclaimable:12604kB kernel_stack:912kB pagetables:15836kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2464 all_unreclaimable? yes
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808284] lowmem_reserve[]: 0 0 0 0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808287] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808297] Node 0 DMA32: 264*4kB 3*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3064kB
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808307] 1448 total pagecache pages
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808308] 0 pages in swap cache
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808310] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808311] Free swap = 0kB
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.808313] Total swap = 0kB
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813860] 985072 pages RAM
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813862] 24998 pages reserved
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813863] 3997 pages shared
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813864] 954326 pages non-shared
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813866] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813875] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813879] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813882] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813885] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813888] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813892] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813895] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813899] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813903] [ 790] 101 790 63477 650 1 0 0 rsyslogd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813906] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813909] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813913] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813916] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813919] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813922] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813925] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813928] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813931] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813934] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813938] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813941] [ 1215] 1002 1215 18359 200 0 0 0 sshd
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813944] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813947] [ 1217] 0 1217 2786 86 0 0 0 muppet
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813950] [ 1255] 0 1255 53179 27291 0 0 0 puppet
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813953] [ 1256] 0 1256 1469 25 1 0 0 tee
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813956] [ 1257] 0 1257 2786 85 1 0 0 muppet
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813959] [ 1258] 0 1258 2554 42 1 0 0 sed
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813962] [ 1259] 0 1259 1475 24 1 0 0 tr
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813965] [ 1260] 0 1260 1540 27 0 0 0 xargs
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813969] [ 3502] 1012 3502 69878 5705 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813972] [ 3504] 1012 3504 68832 6172 1 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813975] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813978] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813981] [ 3533] 1012 3533 68271 5618 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813984] [ 3691] 1012 3691 470971 408066 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813987] [ 3867] 1012 3867 380918 318043 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813990] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813993] [ 5213] 1012 5213 140498 74388 1 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813996] [ 6211] 1012 6211 108352 40044 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.813999] [ 6271] 1012 6271 1099 25 1 0 0 sh
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814002] [ 6277] 1012 6277 68424 5870 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814005] [ 6563] 0 6563 89584 2536 1 0 0 php5-fpm
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814008] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814011] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814014] [ 6658] 1012 6658 1099 25 0 0 0 sh
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814017] [ 6659] 1012 6659 68359 5797 1 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814020] [ 6661] 1012 6661 1099 24 0 0 0 sh
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814023] [ 6662] 1012 6662 68294 5747 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814026] [ 6666] 1012 6666 68169 5589 1 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814029] [ 6667] 1012 6667 1099 25 0 0 0 sh
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814032] [ 6668] 1012 6668 68294 5746 0 0 0 php
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814035] Out of memory: Kill process 3691 (php) score 425 or sacrifice child
Jul 16 08:49:52 ip-10-61-174-46 kernel: [1491487.814046] Killed process 3691 (php) total-vm:1883884kB, anon-rss:1632264kB, file-rss:0kB
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824134] bundle invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824139] bundle cpuset=/ mems_allowed=0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824143] Pid: 6966, comm: bundle Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824145] Call Trace:
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824159] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824169] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824172] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824177] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824182] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824185] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824193] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824195] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824198] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824202] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824205] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824206] Mem-Info:
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824208] Node 0 DMA per-cpu:
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824209] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824211] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824212] Node 0 DMA32 per-cpu:
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824213] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824215] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824217] active_anon:931450 inactive_anon:1395 isolated_anon:0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824218] active_file:0 inactive_file:14 isolated_file:0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824219] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824219] free:2835 slab_reclaimable:2461 slab_unreclaimable:3137
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824220] mapped:1354 shmem:1406 pagetables:3902 bounce:0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824222] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824228] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824231] Node 0 DMA32 free:3004kB min:7848kB low:9808kB high:11772kB active_anon:3725800kB inactive_anon:5580kB active_file:0kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5624kB slab_reclaimable:9844kB slab_unreclaimable:12548kB kernel_stack:856kB pagetables:15608kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2331 all_unreclaimable? yes
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824238] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824240] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824247] Node 0 DMA32: 176*4kB 26*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3008kB
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824253] 1439 total pagecache pages
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824255] 0 pages in swap cache
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824256] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824257] Free swap = 0kB
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.824258] Total swap = 0kB
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.829995] 985072 pages RAM
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.829998] 24998 pages reserved
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.829999] 3951 pages shared
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830000] 954383 pages non-shared
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830002] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830010] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830014] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830017] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830020] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830023] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830027] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830030] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830033] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830037] [ 790] 101 790 63477 651 1 0 0 rsyslogd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830041] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830044] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830048] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830051] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830054] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830057] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830060] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830063] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830066] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830069] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830073] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830076] [ 1215] 1002 1215 18359 200 0 0 0 sshd
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830079] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830082] [ 1217] 0 1217 2790 90 0 0 0 muppet
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830085] [ 3502] 1012 3502 69878 5708 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830088] [ 3504] 1012 3504 68832 6178 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830091] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830094] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830098] [ 3533] 1012 3533 68271 5618 0 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830101] [ 3867] 1012 3867 602819 539947 0 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830104] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830107] [ 5213] 1012 5213 179956 108582 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830110] [ 6211] 1012 6211 224182 154361 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830113] [ 6563] 0 6563 89584 2520 0 0 0 php5-fpm
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830116] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830119] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830122] [ 6666] 1012 6666 104882 37143 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830125] [ 6749] 1012 6749 93782 28297 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830128] [ 6771] 1012 6771 1099 25 1 0 0 sh
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830131] [ 6772] 1012 6772 68424 5870 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830135] [ 6875] 1012 6875 1099 25 1 0 0 sh
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830138] [ 6877] 1012 6877 68424 5869 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830141] [ 6955] 1012 6955 1099 25 0 0 0 sh
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830144] [ 6956] 1012 6956 68359 5774 1 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830147] [ 6960] 1012 6960 1099 24 0 0 0 sh
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830150] [ 6961] 1012 6961 68260 5715 0 0 0 php
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830153] [ 6966] 0 6966 14001 5170 0 0 0 bundle
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830156] Out of memory: Kill process 3867 (php) score 563 or sacrifice child
Jul 16 08:50:02 ip-10-61-174-46 kernel: [1491497.830166] Killed process 3867 (php) total-vm:2411276kB, anon-rss:2159788kB, file-rss:0kB
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804344] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804348] php cpuset=/ mems_allowed=0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804352] Pid: 6749, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804354] Call Trace:
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804362] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804365] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804368] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804371] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804375] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804379] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804382] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804386] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804389] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804393] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804397] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804400] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804404] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804407] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804410] Mem-Info:
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804411] Node 0 DMA per-cpu:
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804414] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804415] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804417] Node 0 DMA32 per-cpu:
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804419] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804421] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804424] active_anon:930182 inactive_anon:1394 isolated_anon:0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804425] active_file:13 inactive_file:8 isolated_file:0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804426] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804427] free:2822 slab_reclaimable:2579 slab_unreclaimable:3289
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804428] mapped:1355 shmem:1406 pagetables:4461 bounce:0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804430] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804438] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804442] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3720728kB inactive_anon:5576kB active_file:52kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5624kB slab_reclaimable:10316kB slab_unreclaimable:13156kB kernel_stack:1000kB pagetables:17844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:126 all_unreclaimable? yes
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804451] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804454] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804464] Node 0 DMA32: 296*4kB 0*8kB 1*16kB 1*32kB 1*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2960kB
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804473] 1427 total pagecache pages
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804474] 0 pages in swap cache
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804476] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804478] Free swap = 0kB
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.804479] Total swap = 0kB
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810311] 985072 pages RAM
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810314] 24998 pages reserved
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810315] 3936 pages shared
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810316] 954392 pages non-shared
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810318] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810326] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810330] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810333] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810336] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810339] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810343] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810346] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810349] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810353] [ 790] 101 790 63477 651 0 0 0 rsyslogd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810356] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810359] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810362] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810365] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810368] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810371] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810374] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810377] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810380] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810384] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810387] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810390] [ 1215] 1002 1215 18359 200 0 0 0 sshd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810393] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810396] [ 1217] 0 1217 2790 89 0 0 0 muppet
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810399] [ 3502] 1012 3502 69878 5718 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810402] [ 3504] 1012 3504 68832 6179 1 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810405] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810408] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810411] [ 3533] 1012 3533 68271 5618 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810414] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810417] [ 5213] 1012 5213 335667 272781 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810420] [ 6211] 1012 6211 338547 275661 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810423] [ 6563] 0 6563 89584 2539 0 0 0 php5-fpm
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810426] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810429] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810432] [ 6666] 1012 6666 215030 129944 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810435] [ 6749] 1012 6749 196597 119184 1 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810438] [ 6960] 1012 6960 1099 24 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810441] [ 6961] 1012 6961 68577 5974 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810444] [ 6979] 1012 6979 102572 34435 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810447] [ 6995] 1012 6995 1099 25 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810450] [ 6997] 1012 6997 68424 5872 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810453] [ 7008] 0 7008 21111 8039 0 0 0 ruby1.8
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810456] [ 7010] 1012 7010 70973 8366 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810459] [ 7011] 1012 7011 1099 24 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810461] [ 7013] 1012 7013 68424 5872 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810464] [ 7015] 1012 7015 1099 25 1 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810467] [ 7016] 1012 7016 68424 5867 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810470] [ 7018] 0 7018 21597 8548 0 0 0 ruby1.8
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810473] [ 7080] 1012 7080 1099 26 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810476] [ 7081] 1012 7081 4682 58 0 0 0 git
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810479] [ 7086] 1012 7086 68476 5522 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810482] [ 7133] 0 7133 1099 25 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810485] [ 7134] 0 7134 1099 25 1 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810488] [ 7135] 0 7135 1099 25 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810490] [ 7136] 0 7136 1099 26 1 0 0 phd
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810493] [ 7140] 0 7140 68360 5736 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810496] [ 7153] 1012 7153 1099 25 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810499] [ 7154] 1012 7154 4661 53 0 0 0 git
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810502] [ 7155] 1012 7155 65772 3314 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810505] [ 7159] 1012 7159 1099 25 0 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810507] [ 7160] 1012 7160 11721 41 0 0 0 php
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810510] [ 7161] 1012 7161 4647 38 0 0 0 git
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810513] [ 7162] 1012 7162 1099 25 1 0 0 sh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810516] [ 7163] 1012 7163 3886 22 0 0 0 ssh
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810519] Out of memory: Kill process 6211 (php) score 287 or sacrifice child
Jul 16 08:50:12 ip-10-61-174-46 kernel: [1491507.810529] Killed process 6211 (php) total-vm:1354188kB, anon-rss:1102644kB, file-rss:0kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692062] php: page allocation failure: order:0, mode:0x4020
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692067] Pid: 7010, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692070] Call Trace:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692072] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692083] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692088] [<ffffffff81162156>] ? allocate_slab+0x196/0x1a0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692092] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692095] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692098] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692104] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692107] [<ffffffff81597d4d>] ? tcp_send_ack+0x2d/0xe0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692112] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692115] [<ffffffff8105746e>] ? entity_tick+0x11e/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692118] [<ffffffff8106fc13>] ? raise_softirq+0x73/0xe0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692123] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692126] [<ffffffff81166ef4>] __kmalloc_node_track_caller+0x84/0x1b0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692130] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692134] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692137] [<ffffffff81597d4d>] ? tcp_send_ack+0x2d/0xe0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692140] [<ffffffff815379b8>] __alloc_skb+0x78/0x240
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692143] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692145] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692148] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692152] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692155] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692158] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692161] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692164] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692168] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692171] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692175] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692180] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692182] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692186] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692190] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692191] <EOI> [<ffffffff8101234e>] ? xen_spin_unlock+0xe/0x20
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692198] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692201] [<ffffffff8117c837>] ? grab_super_passive+0x47/0xc0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692204] [<ffffffff8117c8fc>] ? prune_super+0x4c/0x1a0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692207] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692210] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692214] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692217] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692220] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692223] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692227] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692230] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692234] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692237] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692240] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692243] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692246] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692250] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692253] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692256] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692260] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692262] Mem-Info:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692263] Node 0 DMA per-cpu:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692266] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692268] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692270] Node 0 DMA32 per-cpu:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692272] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692274] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692278] active_anon:930847 inactive_anon:1394 isolated_anon:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692279] active_file:6 inactive_file:30 isolated_file:26
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692280] unevictable:0 dirty:4 writeback:0 unstable:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692281] free:2822 slab_reclaimable:2549 slab_unreclaimable:3197
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692282] mapped:1388 shmem:1406 pagetables:4217 bounce:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692285] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692294] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692298] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3723388kB inactive_anon:5576kB active_file:24kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):104kB present:3862656kB mlocked:0kB dirty:16kB writeback:0kB mapped:5552kB shmem:5624kB slab_reclaimable:10196kB slab_unreclaimable:12788kB kernel_stack:952kB pagetables:16868kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692308] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692312] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692321] Node 0 DMA32: 338*4kB 0*8kB 0*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3016kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692330] 1423 total pagecache pages
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692332] 0 pages in swap cache
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692334] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692336] Free swap = 0kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692337] Total swap = 0kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692339] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692341] cache: kmalloc-1024, object size: 1024, buffer size: 1024, default order: 2, min order: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491512.692344] node 0: slabs: 119, objs: 1904, free: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812148] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812153] php cpuset=/ mems_allowed=0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812156] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812158] Call Trace:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812166] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812172] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812175] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812180] [<ffffffff81312aa6>] ? radix_tree_next_hole+0x36/0x50
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812184] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812189] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812191] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812195] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812199] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812203] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812206] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812209] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812213] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812217] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812221] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812225] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812229] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812231] Mem-Info:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812232] Node 0 DMA per-cpu:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812235] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812236] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812238] Node 0 DMA32 per-cpu:
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812240] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812242] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812245] active_anon:930884 inactive_anon:1394 isolated_anon:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812246] active_file:0 inactive_file:29 isolated_file:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812247] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812248] free:2822 slab_reclaimable:2530 slab_unreclaimable:3202
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812249] mapped:1353 shmem:1406 pagetables:4193 bounce:0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812251] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812260] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812264] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3723536kB inactive_anon:5576kB active_file:0kB inactive_file:116kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5624kB slab_reclaimable:10120kB slab_unreclaimable:12808kB kernel_stack:904kB pagetables:16772kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:673 all_unreclaimable? yes
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812272] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812275] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812285] Node 0 DMA32: 326*4kB 0*8kB 1*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2952kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812294] 1425 total pagecache pages
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812295] 0 pages in swap cache
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812297] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812299] Free swap = 0kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.812300] Total swap = 0kB
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817794] 985072 pages RAM
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817796] 24998 pages reserved
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817797] 3944 pages shared
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817798] 954404 pages non-shared
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817800] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817807] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817810] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817813] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817815] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817817] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817819] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817821] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817824] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817827] [ 790] 101 790 63477 651 0 0 0 rsyslogd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817829] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817832] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817834] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817836] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817838] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817841] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817843] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817845] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817847] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817849] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817852] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817854] [ 1215] 1002 1215 18359 200 0 0 0 sshd
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817856] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817858] [ 1217] 0 1217 2790 89 0 0 0 muppet
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817860] [ 3502] 1012 3502 69878 5729 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817862] [ 3504] 1012 3504 68832 6178 0 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817865] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817867] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817869] [ 3533] 1012 3533 68271 5618 0 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817871] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817873] [ 5213] 1012 5213 389686 326746 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817875] [ 6563] 0 6563 89584 2523 1 0 0 php5-fpm
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817878] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817880] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817882] [ 6666] 1012 6666 277807 214890 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817884] [ 6749] 1012 6749 274927 212049 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817886] [ 6979] 1012 6979 137994 71239 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817888] [ 6995] 1012 6995 1099 25 0 0 0 sh
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817891] [ 6997] 1012 6997 68642 6036 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817893] [ 7008] 0 7008 21111 8039 0 0 0 ruby1.8
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817895] [ 7010] 1012 7010 76951 14345 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817897] [ 7011] 1012 7011 1099 24 0 0 0 sh
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817899] [ 7013] 1012 7013 68424 5874 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817901] [ 7015] 1012 7015 1099 25 1 0 0 sh
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817903] [ 7016] 1012 7016 68424 5870 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817905] [ 7018] 0 7018 21596 8547 1 0 0 ruby1.8
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817907] [ 7160] 1012 7160 68169 5594 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817909] [ 7164] 1012 7164 1099 25 0 0 0 sh
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817911] [ 7169] 1012 7169 5216 118 0 0 0 git
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817913] [ 7173] 1012 7173 68411 5459 0 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817915] [ 7175] 1012 7175 1099 25 1 0 0 sh
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817917] [ 7176] 1012 7176 68359 5797 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817919] [ 7218] 1012 7218 68169 5589 1 0 0 php
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817922] Out of memory: Kill process 5213 (php) score 341 or sacrifice child
Jul 16 08:50:19 ip-10-61-174-46 kernel: [1491514.817932] Killed process 5213 (php) total-vm:1558744kB, anon-rss:1306984kB, file-rss:0kB
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263123] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263127] php cpuset=/ mems_allowed=0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263129] Pid: 7010, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263131] Call Trace:
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263138] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263143] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263146] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263149] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263152] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263154] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263157] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263162] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263164] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263170] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263172] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263175] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263178] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263181] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263184] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263186] Mem-Info:
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263187] Node 0 DMA per-cpu:
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263189] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263190] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263191] Node 0 DMA32 per-cpu:
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263193] CPU 0: hi: 186, btch: 31 usd: 31
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263194] CPU 1: hi: 186, btch: 31 usd: 61
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263197] active_anon:930992 inactive_anon:1394 isolated_anon:0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263198] active_file:0 inactive_file:0 isolated_file:19
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263198] unevictable:0 dirty:14 writeback:0 unstable:0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263199] free:2909 slab_reclaimable:2522 slab_unreclaimable:3145
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263200] mapped:1351 shmem:1406 pagetables:4117 bounce:0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263201] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263208] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263211] Node 0 DMA32 free:3300kB min:7848kB low:9808kB high:11772kB active_anon:3723968kB inactive_anon:5576kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:56kB writeback:0kB mapped:5404kB shmem:5624kB slab_reclaimable:10088kB slab_unreclaimable:12580kB kernel_stack:904kB pagetables:16468kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:115 all_unreclaimable? yes
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263217] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263219] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263226] Node 0 DMA32: 329*4kB 4*8kB 0*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3300kB
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263233] 1444 total pagecache pages
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263234] 0 pages in swap cache
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263235] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263236] Free swap = 0kB
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.263237] Total swap = 0kB
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268956] 985072 pages RAM
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268958] 24998 pages reserved
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268960] 3963 pages shared
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268961] 954221 pages non-shared
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268963] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268972] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268975] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268978] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268982] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268985] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268988] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268991] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268995] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.268999] [ 790] 101 790 63477 651 0 0 0 rsyslogd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269002] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269005] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269008] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269012] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269015] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269018] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269021] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269024] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269027] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269031] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269035] [ 973] 0 973 18359 199 0 0 0 sshd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269038] [ 1215] 1002 1215 18359 200 1 0 0 sshd
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269041] [ 1216] 1002 1216 10475 91 0 0 0 sudo
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269044] [ 1217] 0 1217 2790 89 0 0 0 muppet
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269047] [ 3502] 1012 3502 69878 5732 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269050] [ 3504] 1012 3504 68832 6180 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269053] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269056] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269059] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269062] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269065] [ 6563] 0 6563 89584 2507 1 0 0 php5-fpm
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269068] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269071] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269075] [ 6666] 1012 6666 331891 268983 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269078] [ 6749] 1012 6749 342707 279806 0 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269081] [ 6979] 1012 6979 266544 203674 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269084] [ 7008] 0 7008 21111 8039 0 0 0 ruby1.8
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269086] [ 7010] 1012 7010 93002 27781 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269090] [ 7018] 0 7018 22084 9028 0 0 0 ruby1.8
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269093] [ 7160] 1012 7160 80230 17601 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269096] [ 7175] 1012 7175 1099 25 1 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269099] [ 7176] 1012 7176 68424 5872 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269102] [ 7218] 1012 7218 122286 53049 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269105] [ 7293] 1012 7293 73376 10743 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269108] [ 7347] 1012 7347 1099 25 0 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269111] [ 7353] 1012 7353 68359 5795 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269113] [ 7469] 1012 7469 1099 24 0 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269116] [ 7470] 1012 7470 67285 4711 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269119] [ 7533] 1012 7533 1099 25 0 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269122] [ 7534] 1012 7534 43716 917 1 0 0 php
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269125] [ 7535] 0 7535 1099 25 0 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269128] [ 7536] 0 7536 1099 25 1 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269131] [ 7537] 0 7537 1099 25 0 0 0 sh
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269134] [ 7538] 0 7538 1902 67 1 0 0 dpkg-query
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269137] [ 7539] 0 7539 2152 24 1 0 0 sed
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269140] Out of memory: Kill process 6749 (php) score 292 or sacrifice child
Jul 16 08:50:24 ip-10-61-174-46 kernel: [1491520.269151] Killed process 6749 (php) total-vm:1370828kB, anon-rss:1119224kB, file-rss:0kB
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832130] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832135] php cpuset=/ mems_allowed=0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832138] Pid: 7160, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832140] Call Trace:
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832159] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832163] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832167] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832170] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832174] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832179] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832182] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832186] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832189] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832195] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832203] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832207] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832209] Mem-Info:
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832210] Node 0 DMA per-cpu:
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832216] Node 0 DMA32 per-cpu:
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832220] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832224] active_anon:931333 inactive_anon:1393 isolated_anon:0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832225] active_file:0 inactive_file:28 isolated_file:0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832226] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832226] free:2822 slab_reclaimable:2403 slab_unreclaimable:3113
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832227] mapped:1352 shmem:1405 pagetables:4158 bounce:0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832230] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832238] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832242] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3725332kB inactive_anon:5572kB active_file:0kB inactive_file:112kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9612kB slab_unreclaimable:12452kB kernel_stack:880kB pagetables:16632kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:612 all_unreclaimable? yes
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832250] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832254] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832263] Node 0 DMA32: 220*4kB 9*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2952kB
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832272] 1423 total pagecache pages
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832274] 0 pages in swap cache
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832276] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832278] Free swap = 0kB
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.832279] Total swap = 0kB
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837727] 985072 pages RAM
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837729] 24998 pages reserved
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837730] 3775 pages shared
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837731] 954570 pages non-shared
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837733] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837743] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837747] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837750] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837753] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837756] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837760] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837763] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837766] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837770] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837773] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837776] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837780] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837783] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837786] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837789] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837792] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837795] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837798] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837801] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837805] [ 3502] 1012 3502 69878 5730 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837808] [ 3504] 1012 3504 68832 6180 1 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837811] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837814] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837817] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837820] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837823] [ 6563] 0 6563 89584 2491 0 0 0 php5-fpm
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837827] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837830] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837833] [ 6666] 1012 6666 432889 370004 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837836] [ 6979] 1012 6979 394103 331171 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837839] [ 7010] 1012 7010 97952 30729 1 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837842] [ 7160] 1012 7160 102080 33875 1 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837845] [ 7218] 1012 7218 136760 69454 1 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837848] [ 7293] 1012 7293 89946 25819 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837851] [ 7347] 1012 7347 1099 25 0 0 0 sh
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837854] [ 7353] 1012 7353 68424 5872 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837857] [ 7469] 1012 7469 1099 24 0 0 0 sh
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837860] [ 7470] 1012 7470 68424 5871 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837863] [ 7533] 1012 7533 1099 25 0 0 0 sh
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837866] [ 7534] 1012 7534 68359 5798 1 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837869] [ 7562] 1012 7562 68169 5590 1 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837872] [ 7568] 1012 7568 1099 25 1 0 0 sh
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837875] [ 7569] 1012 7569 4665 53 0 0 0 git
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837878] [ 7570] 1012 7570 68476 5481 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837881] [ 7576] 1012 7576 1099 25 1 0 0 sh
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837884] [ 7577] 1012 7577 4709 62 1 0 0 git
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837887] [ 7578] 1012 7578 66696 4223 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837890] [ 7580] 1012 7580 1099 24 0 0 0 sh
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837893] [ 7581] 1012 7581 65043 3031 0 0 0 php
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837896] Out of memory: Kill process 6666 (php) score 386 or sacrifice child
Jul 16 08:50:32 ip-10-61-174-46 kernel: [1491527.837907] Killed process 6666 (php) total-vm:1731556kB, anon-rss:1480016kB, file-rss:0kB
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880129] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880134] php cpuset=/ mems_allowed=0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880137] Pid: 7634, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880139] Call Trace:
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880147] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880157] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880161] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880166] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880169] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880173] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880176] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880180] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880183] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880187] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880191] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880196] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880203] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880207] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880209] Mem-Info:
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880211] Node 0 DMA per-cpu:
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880217] Node 0 DMA32 per-cpu:
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880219] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880221] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880224] active_anon:931310 inactive_anon:1393 isolated_anon:0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880225] active_file:0 inactive_file:0 isolated_file:0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880226] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880227] free:2956 slab_reclaimable:2454 slab_unreclaimable:3116
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880228] mapped:1352 shmem:1405 pagetables:3996 bounce:0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880230] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880239] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880243] Node 0 DMA32 free:3488kB min:7848kB low:9808kB high:11772kB active_anon:3725240kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9816kB slab_unreclaimable:12464kB kernel_stack:864kB pagetables:15984kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880251] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880254] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880264] Node 0 DMA32: 414*4kB 3*8kB 0*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3536kB
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880273] 1423 total pagecache pages
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880275] 0 pages in swap cache
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880276] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880278] Free swap = 0kB
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.880279] Total swap = 0kB
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885692] 985072 pages RAM
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885695] 24998 pages reserved
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885696] 3780 pages shared
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885697] 954425 pages non-shared
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885699] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885708] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885711] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885714] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885717] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885720] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885724] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885727] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885731] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885735] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885738] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885741] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885744] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885747] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885750] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885753] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885756] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885759] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885762] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885766] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885769] [ 3502] 1012 3502 69878 5730 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885772] [ 3504] 1012 3504 68832 6180 0 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885775] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885778] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885781] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885784] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885788] [ 6563] 0 6563 89584 2527 0 0 0 php5-fpm
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885791] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885794] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885797] [ 6979] 1012 6979 542528 479631 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885800] [ 7010] 1012 7010 105902 38034 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885803] [ 7160] 1012 7160 135044 66849 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885806] [ 7218] 1012 7218 275375 212449 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885809] [ 7293] 1012 7293 126666 57143 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885812] [ 7469] 1012 7469 1099 24 0 0 0 sh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885815] [ 7470] 1012 7470 68776 6177 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885818] [ 7533] 1012 7533 1099 25 0 0 0 sh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885821] [ 7534] 1012 7534 68642 6038 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885824] [ 7562] 1012 7562 75929 13316 0 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885827] [ 7580] 1012 7580 1099 24 0 0 0 sh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885830] [ 7581] 1012 7581 68424 5874 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885833] [ 7625] 1012 7625 1099 26 0 0 0 sh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885836] [ 7626] 1012 7626 4652 58 0 0 0 git
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885839] [ 7627] 1012 7627 68476 5523 1 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885842] [ 7633] 1012 7633 1099 25 0 0 0 sh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885845] [ 7634] 1012 7634 68260 5715 0 0 0 php
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885848] [ 7661] 1012 7661 1099 25 1 0 0 sh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885851] [ 7662] 1012 7662 10901 166 1 0 0 ssh
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885854] Out of memory: Kill process 6979 (php) score 500 or sacrifice child
Jul 16 08:50:38 ip-10-61-174-46 kernel: [1491533.885865] Killed process 6979 (php) total-vm:2170112kB, anon-rss:1918524kB, file-rss:0kB
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816133] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816137] php cpuset=/ mems_allowed=0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816139] Pid: 7562, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816141] Call Trace:
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816153] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816156] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816159] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816162] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816164] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816168] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816170] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816173] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816177] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816179] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816182] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816184] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816186] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816189] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816191] Mem-Info:
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816192] Node 0 DMA per-cpu:
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816194] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816195] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816196] Node 0 DMA32 per-cpu:
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816198] CPU 0: hi: 186, btch: 31 usd: 119
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816199] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816202] active_anon:931933 inactive_anon:1393 isolated_anon:0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816203] active_file:9 inactive_file:23 isolated_file:12
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816203] unevictable:0 dirty:0 writeback:1 unstable:0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816204] free:2787 slab_reclaimable:2321 slab_unreclaimable:3036
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816205] mapped:1371 shmem:1405 pagetables:3830 bounce:0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816206] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816213] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816216] Node 0 DMA32 free:2812kB min:7848kB low:9808kB high:11772kB active_anon:3727732kB inactive_anon:5572kB active_file:36kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):48kB present:3862656kB mlocked:0kB dirty:0kB writeback:4kB mapped:5484kB shmem:5620kB slab_reclaimable:9284kB slab_unreclaimable:12144kB kernel_stack:832kB pagetables:15320kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:259 all_unreclaimable? yes
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816222] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816225] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816232] Node 0 DMA32: 141*4kB 29*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2812kB
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816238] 1424 total pagecache pages
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816240] 0 pages in swap cache
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816241] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816242] Free swap = 0kB
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.816243] Total swap = 0kB
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821743] 985072 pages RAM
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821745] 24998 pages reserved
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821747] 3792 pages shared
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821748] 954461 pages non-shared
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821750] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821758] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821762] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821765] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821768] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821771] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821774] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821778] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821781] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821785] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821789] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821792] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821795] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821798] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821801] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821804] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821808] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821811] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821814] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821817] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821821] [ 3502] 1012 3502 69878 5732 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821824] [ 3504] 1012 3504 68832 6188 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821826] [ 3506] 1012 3506 69239 6592 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821829] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821833] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821836] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821839] [ 6563] 0 6563 89584 2511 1 0 0 php5-fpm
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821842] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821845] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821849] [ 7010] 1012 7010 112780 43621 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821852] [ 7160] 1012 7160 152343 89327 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821854] [ 7218] 1012 7218 681928 619063 0 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821857] [ 7293] 1012 7293 143566 79491 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821860] [ 7562] 1012 7562 103550 35532 0 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821863] [ 7666] 1012 7666 1099 24 1 0 0 sh
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821866] [ 7667] 1012 7667 68424 5871 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821869] [ 7669] 1012 7669 1099 25 1 0 0 sh
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821872] [ 7670] 1012 7670 68424 5875 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821875] [ 7672] 1012 7672 69955 7311 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821878] [ 7682] 1012 7682 1099 24 1 0 0 sh
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821881] [ 7683] 1012 7683 68359 5797 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821884] [ 7708] 1012 7708 1099 25 1 0 0 sh
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821887] [ 7709] 1012 7709 68359 5796 1 0 0 php
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821890] Out of memory: Kill process 7218 (php) score 646 or sacrifice child
Jul 16 08:50:48 ip-10-61-174-46 kernel: [1491543.821901] Killed process 7218 (php) total-vm:2727712kB, anon-rss:2476244kB, file-rss:8kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915544] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915548] php cpuset=/ mems_allowed=0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915551] Pid: 7823, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915553] Call Trace:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915560] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915562] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915564] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915567] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915570] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915574] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915576] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915579] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915584] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915586] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915590] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915592] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915594] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915597] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915601] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915603] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915606] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915609] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915610] Mem-Info:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915612] Node 0 DMA per-cpu:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915614] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915615] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915616] Node 0 DMA32 per-cpu:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915618] CPU 0: hi: 186, btch: 31 usd: 51
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915619] CPU 1: hi: 186, btch: 31 usd: 3
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915622] active_anon:931088 inactive_anon:1393 isolated_anon:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915622] active_file:24 inactive_file:17 isolated_file:22
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915623] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915624] free:3269 slab_reclaimable:2322 slab_unreclaimable:3068
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915624] mapped:1348 shmem:1405 pagetables:4133 bounce:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915626] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915633] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915636] Node 0 DMA32 free:4740kB min:7848kB low:9808kB high:11772kB active_anon:3724352kB inactive_anon:5572kB active_file:96kB inactive_file:68kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5392kB shmem:5620kB slab_reclaimable:9288kB slab_unreclaimable:12272kB kernel_stack:880kB pagetables:16532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1048 all_unreclaimable? yes
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915642] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915644] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915651] Node 0 DMA32: 225*4kB 48*8kB 42*16kB 17*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 4740kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915658] 1454 total pagecache pages
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915659] 0 pages in swap cache
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915661] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915662] Free swap = 0kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.915662] Total swap = 0kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921308] 985072 pages RAM
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921310] 24998 pages reserved
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921312] 3777 pages shared
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921313] 954067 pages non-shared
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921315] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921324] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921327] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921330] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921334] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921337] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921341] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921344] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921347] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921351] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921354] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921358] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921361] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921364] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921367] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921370] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921373] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921377] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921380] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921383] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921387] [ 3502] 1012 3502 69878 5731 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921390] [ 3504] 1012 3504 68832 6188 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921393] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921396] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921399] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921402] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921405] [ 6563] 0 6563 89584 2495 1 0 0 php5-fpm
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921408] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921411] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921414] [ 7010] 1012 7010 121434 52378 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921417] [ 7160] 1012 7160 471793 408938 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921420] [ 7293] 1012 7293 373621 310729 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921424] [ 7562] 1012 7562 137498 70552 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921427] [ 7672] 1012 7672 72628 9995 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921430] [ 7721] 1012 7721 72288 9679 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921433] [ 7784] 1012 7784 1099 25 0 0 0 sh
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921436] [ 7785] 1012 7785 68424 5871 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921439] [ 7787] 1012 7787 69327 6780 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921442] [ 7797] 1012 7797 1099 24 0 0 0 sh
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921445] [ 7798] 1012 7798 68424 5871 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921448] [ 7814] 1012 7814 1099 25 0 0 0 sh
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921451] [ 7815] 1012 7815 68359 5797 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921454] [ 7821] 1012 7821 1099 25 0 0 0 sh
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921457] [ 7822] 1012 7822 4736 66 1 0 0 git
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921459] [ 7823] 1012 7823 67216 4281 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921463] [ 7824] 1012 7824 1099 26 0 0 0 sh
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921466] [ 7825] 1012 7825 66830 4300 1 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921469] [ 7832] 1012 7832 1099 25 1 0 0 sh
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921472] [ 7833] 1012 7833 4655 53 0 0 0 git
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921474] [ 7834] 1012 7834 64041 1436 0 0 0 php
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921477] Out of memory: Kill process 7160 (php) score 426 or sacrifice child
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.921489] Killed process 7160 (php) total-vm:1887172kB, anon-rss:1635752kB, file-rss:0kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980075] php: page allocation failure: order:0, mode:0x201da
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980079] Pid: 7160, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980081] Call Trace:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980092] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980096] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980101] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980105] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980108] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980113] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980115] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980119] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980123] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980128] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980132] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980135] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980138] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980143] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980146] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980150] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980153] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980155] Mem-Info:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980157] Node 0 DMA per-cpu:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980159] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980161] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980162] Node 0 DMA32 per-cpu:
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980165] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980168] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980172] active_anon:931088 inactive_anon:1393 isolated_anon:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980173] active_file:24 inactive_file:42 isolated_file:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980173] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980174] free:2900 slab_reclaimable:2322 slab_unreclaimable:3168
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980175] mapped:1348 shmem:1405 pagetables:4133 bounce:0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980177] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980186] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980189] Node 0 DMA32 free:3264kB min:7848kB low:9808kB high:11772kB active_anon:3724352kB inactive_anon:5572kB active_file:96kB inactive_file:168kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5392kB shmem:5620kB slab_reclaimable:9288kB slab_unreclaimable:12672kB kernel_stack:880kB pagetables:16532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980198] lowmem_reserve[]: 0 0 0 0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980201] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980212] Node 0 DMA32: 324*4kB 0*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3264kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980221] 1454 total pagecache pages
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980223] 0 pages in swap cache
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980225] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980226] Free swap = 0kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.980227] Total swap = 0kB
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.987425] 985072 pages RAM
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.987428] 24998 pages reserved
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.987429] 3777 pages shared
Jul 16 08:50:58 ip-10-61-174-46 kernel: [1491553.987430] 954490 pages non-shared
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980777] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980781] php cpuset=/ mems_allowed=0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980785] Pid: 7918, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980787] Call Trace:
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980795] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980799] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980802] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980806] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980810] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980815] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980817] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980821] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980826] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980829] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980834] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980837] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980840] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980846] [<ffffffff8164a855>] ? __slab_free+0x1d2/0x2f3
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980850] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980855] [<ffffffff81163774>] ? kmem_cache_free+0x104/0x110
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980858] [<ffffffff81142bb5>] ? remove_vma+0x65/0x80
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980862] [<ffffffff81143e23>] ? do_munmap+0x1f3/0x2f0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980866] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980868] Mem-Info:
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980870] Node 0 DMA per-cpu:
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980872] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980874] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980876] Node 0 DMA32 per-cpu:
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980878] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980880] CPU 1: hi: 186, btch: 31 usd: 6
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980883] active_anon:931897 inactive_anon:1392 isolated_anon:0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980884] active_file:20 inactive_file:19 isolated_file:20
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980885] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980886] free:4190 slab_reclaimable:2399 slab_unreclaimable:2756
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980887] mapped:1384 shmem:1405 pagetables:3764 bounce:0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980889] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980898] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980902] Node 0 DMA32 free:8300kB min:7848kB low:9808kB high:11772kB active_anon:3727588kB inactive_anon:5568kB active_file:80kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5536kB shmem:5620kB slab_reclaimable:9596kB slab_unreclaimable:11024kB kernel_stack:824kB pagetables:15056kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980911] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980914] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980924] Node 0 DMA32: 341*4kB 179*8kB 118*16kB 39*32kB 5*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8300kB
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980933] 1482 total pagecache pages
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980934] 0 pages in swap cache
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980936] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980937] Free swap = 0kB
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.980939] Total swap = 0kB
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988434] 985072 pages RAM
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988436] 24998 pages reserved
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988437] 3854 pages shared
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988438] 953583 pages non-shared
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988440] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988449] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988452] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988456] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988459] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988462] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988465] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988468] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988471] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988474] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988477] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988480] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988483] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988486] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988488] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988491] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988493] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988496] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988499] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988502] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988506] [ 3502] 1012 3502 69878 5733 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988509] [ 3504] 1012 3504 68832 6188 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988511] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988514] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988517] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988519] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988522] [ 6563] 0 6563 89584 2527 0 0 0 php5-fpm
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988525] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988527] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988530] [ 7010] 1012 7010 130314 61594 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988533] [ 7293] 1012 7293 573890 511005 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988536] [ 7562] 1012 7562 308657 245758 0 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988539] [ 7672] 1012 7672 77117 14519 0 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988541] [ 7721] 1012 7721 95662 29451 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988544] [ 7787] 1012 7787 85888 22821 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988547] [ 7824] 1012 7824 1099 26 0 0 0 sh
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988550] [ 7825] 1012 7825 68508 5912 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988553] [ 7907] 1012 7907 1099 24 0 0 0 sh
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988556] [ 7912] 1012 7912 68359 5774 1 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988568] [ 7918] 1012 7918 31058 415 0 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988572] [ 7919] 1012 7919 1099 25 0 0 0 sh
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988574] [ 7920] 1012 7920 31058 416 0 0 0 php
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988577] [ 7923] 1012 7923 1099 26 1 0 0 sh
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988590] [ 7924] 1012 7924 4647 41 0 0 0 git
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988593] Out of memory: Kill process 7293 (php) score 533 or sacrifice child
Jul 16 08:51:05 ip-10-61-174-46 kernel: [1491560.988605] Killed process 7293 (php) total-vm:2295560kB, anon-rss:2044004kB, file-rss:16kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116021] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116025] ntpd cpuset=/ mems_allowed=0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116028] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116030] Call Trace:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116037] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116039] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116041] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116044] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116047] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116053] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116066] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116071] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116074] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116078] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116081] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116085] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116089] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116093] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116097] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116100] Mem-Info:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116101] Node 0 DMA per-cpu:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116104] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116106] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116107] Node 0 DMA32 per-cpu:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116110] CPU 0: hi: 186, btch: 31 usd: 53
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116111] CPU 1: hi: 186, btch: 31 usd: 51
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116115] active_anon:931847 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116116] active_file:12 inactive_file:8 isolated_file:29
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116117] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116118] free:2816 slab_reclaimable:2371 slab_unreclaimable:3037
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116119] mapped:1399 shmem:1405 pagetables:3806 bounce:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116121] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116129] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116133] Node 0 DMA32 free:2928kB min:7848kB low:9808kB high:11772kB active_anon:3727388kB inactive_anon:5572kB active_file:48kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):116kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5596kB shmem:5620kB slab_reclaimable:9484kB slab_unreclaimable:12148kB kernel_stack:816kB pagetables:15224kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:237 all_unreclaimable? yes
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116142] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116145] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116156] Node 0 DMA32: 52*4kB 14*8kB 13*16kB 5*32kB 4*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2992kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116165] 1435 total pagecache pages
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116167] 0 pages in swap cache
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116169] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116170] Free swap = 0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.116172] Total swap = 0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121626] 985072 pages RAM
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121628] 24998 pages reserved
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121629] 3788 pages shared
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121629] 954452 pages non-shared
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121631] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121638] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121641] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121643] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121645] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121648] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121650] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121653] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121655] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121658] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121660] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121663] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121665] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121667] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121669] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121671] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121673] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121676] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121678] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121680] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121683] [ 3502] 1012 3502 69878 5732 0 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121685] [ 3504] 1012 3504 68832 6188 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121687] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121689] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121691] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121694] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121696] [ 6563] 0 6563 89584 2511 1 0 0 php5-fpm
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121698] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121700] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121703] [ 7010] 1012 7010 143566 79457 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121705] [ 7562] 1012 7562 661191 598300 0 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121707] [ 7672] 1012 7672 93260 27961 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121709] [ 7721] 1012 7721 130304 61638 0 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121711] [ 7787] 1012 7787 145636 82261 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121713] [ 7918] 1012 7918 90890 26383 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121715] [ 7995] 1012 7995 69875 7242 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121717] [ 8057] 1012 8057 1099 24 0 0 0 sh
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121719] [ 8058] 1012 8058 68260 5715 0 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121721] [ 8060] 1012 8060 1099 24 0 0 0 sh
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121724] [ 8061] 1012 8061 66830 4299 1 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121726] [ 8063] 1012 8063 1099 24 0 0 0 sh
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121728] [ 8064] 1012 8064 66797 4284 0 0 0 php
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121730] Out of memory: Kill process 7562 (php) score 624 or sacrifice child
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.121740] Killed process 7562 (php) total-vm:2644764kB, anon-rss:2393200kB, file-rss:0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124222] php: page allocation failure: order:0, mode:0x201da
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124226] Pid: 7562, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124228] Call Trace:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124237] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124241] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124246] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124249] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124253] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124258] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124261] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124265] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124269] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124272] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124276] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124279] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124283] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124287] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124290] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124293] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124297] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124300] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124302] Mem-Info:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124304] Node 0 DMA per-cpu:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124306] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124308] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124309] Node 0 DMA32 per-cpu:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124312] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124313] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124317] active_anon:931847 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124317] active_file:12 inactive_file:8 isolated_file:29
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124318] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124319] free:2936 slab_reclaimable:2371 slab_unreclaimable:3037
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124320] mapped:1399 shmem:1405 pagetables:3806 bounce:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124322] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124331] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124335] Node 0 DMA32 free:3408kB min:7848kB low:9808kB high:11772kB active_anon:3727388kB inactive_anon:5572kB active_file:48kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):116kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5596kB shmem:5620kB slab_reclaimable:9484kB slab_unreclaimable:12148kB kernel_stack:816kB pagetables:15224kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:28 all_unreclaimable? no
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124344] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124347] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124357] Node 0 DMA32: 112*4kB 18*8kB 12*16kB 8*32kB 5*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3408kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124367] 1435 total pagecache pages
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124368] 0 pages in swap cache
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124370] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124371] Free swap = 0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.124373] Total swap = 0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.129579] 985072 pages RAM
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.129581] 24998 pages reserved
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.129582] 3787 pages shared
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.129583] 954452 pages non-shared
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216063] ntpd: page allocation failure: order:0, mode:0x4020
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216068] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216070] Call Trace:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216072] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216085] [<ffffffff813ac291>] ? notify_remote_via_irq+0x31/0x50
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216088] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216092] [<ffffffff81162156>] ? allocate_slab+0x196/0x1a0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216094] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216097] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216099] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216104] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216107] [<ffffffff81057135>] ? reweight_entity+0x165/0x180
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216110] [<ffffffff81597d4d>] ? tcp_send_ack+0x2d/0xe0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216113] [<ffffffff810573bf>] ? entity_tick+0x6f/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216117] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216120] [<ffffffff811119fe>] ? perf_event_task_tick+0x1e/0xa0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216123] [<ffffffff81166ef4>] __kmalloc_node_track_caller+0x84/0x1b0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216126] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216129] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216131] [<ffffffff81597d4d>] ? tcp_send_ack+0x2d/0xe0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216133] [<ffffffff815379b8>] __alloc_skb+0x78/0x240
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216135] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216137] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216139] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216142] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216144] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216148] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216149] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216152] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216155] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216157] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216161] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216164] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216166] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216168] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216170] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216172] <EOI> [<ffffffff813138c6>] ? radix_tree_gang_lookup_tag.part.7+0x16/0xf0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216178] [<ffffffff813139c4>] ? radix_tree_gang_lookup_tag+0x24/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216196] [<ffffffffa00f20e1>] ? xfs_perag_get_tag+0x41/0xf0 [xfs]
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216205] [<ffffffffa00b4d60>] ? xfs_reclaim_inodes_count+0x40/0x60 [xfs]
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216214] [<ffffffffa00b14e5>] ? xfs_fs_nr_cached_objects+0x15/0x20 [xfs]
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216217] [<ffffffff8117c924>] ? prune_super+0x74/0x1a0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216220] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216222] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216225] [<ffffffff8108b8d6>] ? autoremove_wake_function+0x16/0x40
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216227] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216230] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216232] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216235] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216238] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216240] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216242] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216245] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216247] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216250] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216252] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216254] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216257] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216259] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216261] Mem-Info:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216262] Node 0 DMA per-cpu:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216264] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216265] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216266] Node 0 DMA32 per-cpu:
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216268] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216269] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216272] active_anon:931847 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216273] active_file:12 inactive_file:8 isolated_file:29
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216273] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216274] free:2792 slab_reclaimable:2371 slab_unreclaimable:3062
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216275] mapped:1399 shmem:1405 pagetables:3806 bounce:0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216276] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216282] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216285] Node 0 DMA32 free:2832kB min:7848kB low:9808kB high:11772kB active_anon:3727388kB inactive_anon:5572kB active_file:48kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):116kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5596kB shmem:5620kB slab_reclaimable:9484kB slab_unreclaimable:12248kB kernel_stack:816kB pagetables:15224kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:42576 all_unreclaimable? yes
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216292] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216294] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216301] Node 0 DMA32: 106*4kB 9*8kB 8*16kB 3*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2832kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216308] 1435 total pagecache pages
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216309] 0 pages in swap cache
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216310] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216312] Free swap = 0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216312] Total swap = 0kB
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216314] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216316] cache: kmalloc-1024, object size: 1024, buffer size: 1024, default order: 2, min order: 0
Jul 16 08:51:15 ip-10-61-174-46 kernel: [1491571.216318] node 0: slabs: 107, objs: 1712, free: 0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840148] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840153] php cpuset=/ mems_allowed=0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840157] Pid: 7995, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840159] Call Trace:
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840167] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840170] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840173] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840177] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840181] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840185] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840188] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840191] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840195] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840199] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840202] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840206] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840211] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840213] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840217] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840222] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840226] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840228] Mem-Info:
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840229] Node 0 DMA per-cpu:
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840232] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840234] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840235] Node 0 DMA32 per-cpu:
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840237] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840239] CPU 1: hi: 186, btch: 31 usd: 53
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840242] active_anon:931292 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840243] active_file:0 inactive_file:0 isolated_file:0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840244] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840245] free:2824 slab_reclaimable:2356 slab_unreclaimable:3144
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840246] mapped:1359 shmem:1405 pagetables:4116 bounce:0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840248] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840257] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840261] Node 0 DMA32 free:2960kB min:7848kB low:9808kB high:11772kB active_anon:3725168kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5620kB slab_reclaimable:9424kB slab_unreclaimable:12576kB kernel_stack:888kB pagetables:16464kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:50 all_unreclaimable? yes
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840270] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840273] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840283] Node 0 DMA32: 144*4kB 34*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2960kB
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840292] 1430 total pagecache pages
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840294] 0 pages in swap cache
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840295] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840297] Free swap = 0kB
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.840298] Total swap = 0kB
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845800] 985072 pages RAM
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845802] 24998 pages reserved
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845804] 3775 pages shared
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845805] 954514 pages non-shared
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845807] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845815] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845819] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845822] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845825] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845828] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845832] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845835] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845838] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845842] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845845] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845849] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845852] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845855] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845858] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845861] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845864] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845868] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845871] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845874] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845878] [ 3502] 1012 3502 69878 5732 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845881] [ 3504] 1012 3504 68832 6193 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845884] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845887] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845890] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845893] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845896] [ 6563] 0 6563 89584 2495 1 0 0 php5-fpm
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845899] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845902] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845905] [ 7010] 1012 7010 311474 248595 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845908] [ 7672] 1012 7672 132896 64501 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845911] [ 7721] 1012 7721 223148 139986 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845914] [ 7787] 1012 7787 378283 315419 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845917] [ 7918] 1012 7918 129850 60946 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845920] [ 7995] 1012 7995 96230 29790 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845923] [ 8060] 1012 8060 1099 24 0 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845926] [ 8061] 1012 8061 68424 5887 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845929] [ 8063] 1012 8063 1099 24 0 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845932] [ 8064] 1012 8064 68424 5872 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845935] [ 8066] 1012 8066 71252 8631 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845938] [ 8067] 1012 8067 1099 25 1 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845941] [ 8068] 1012 8068 68424 5872 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845944] [ 8071] 1012 8071 1099 25 0 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845947] [ 8072] 1012 8072 68424 5869 1 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845950] [ 8106] 1012 8106 1099 26 0 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845953] [ 8107] 1012 8107 4664 58 0 0 0 git
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845956] [ 8109] 1012 8109 68476 5523 0 0 0 php
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845959] [ 8121] 1012 8121 1099 25 0 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845962] [ 8122] 1012 8122 1099 26 1 0 0 sh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845965] [ 8123] 1012 8123 3886 21 0 0 0 ssh
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845968] [ 8124] 1012 8124 2745 18 1 0 0 git
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845971] Out of memory: Kill process 7787 (php) score 329 or sacrifice child
Jul 16 08:51:24 ip-10-61-174-46 kernel: [1491579.845982] Killed process 7787 (php) total-vm:1513132kB, anon-rss:1261676kB, file-rss:0kB
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972128] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972132] php cpuset=/ mems_allowed=0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972135] Pid: 3502, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972136] Call Trace:
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972143] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972146] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972148] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972151] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972154] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972157] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972159] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972163] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972166] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972168] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972171] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972174] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972176] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972179] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972182] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972186] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972188] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972191] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972194] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972195] Mem-Info:
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972197] Node 0 DMA per-cpu:
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972198] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972200] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972201] Node 0 DMA32 per-cpu:
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972203] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972204] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972206] active_anon:931661 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972207] active_file:30 inactive_file:19 isolated_file:8
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972208] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972208] free:2841 slab_reclaimable:2439 slab_unreclaimable:3060
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972209] mapped:1372 shmem:1405 pagetables:3907 bounce:0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972211] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972217] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972220] Node 0 DMA32 free:3028kB min:7848kB low:9808kB high:11772kB active_anon:3726644kB inactive_anon:5572kB active_file:120kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):32kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5488kB shmem:5620kB slab_reclaimable:9756kB slab_unreclaimable:12240kB kernel_stack:832kB pagetables:15628kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:71698 all_unreclaimable? yes
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972227] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972229] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972236] Node 0 DMA32: 239*4kB 5*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3028kB
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972243] 1473 total pagecache pages
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972244] 0 pages in swap cache
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972245] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972246] Free swap = 0kB
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.972247] Total swap = 0kB
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977881] 985072 pages RAM
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977883] 24998 pages reserved
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977884] 3789 pages shared
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977886] 954543 pages non-shared
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977887] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977896] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977900] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977903] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977906] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977909] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977913] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977916] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977919] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977923] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977927] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977930] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977933] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977936] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977939] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977942] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977945] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977948] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977951] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977955] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977958] [ 3502] 1012 3502 69878 5733 0 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977961] [ 3504] 1012 3504 68832 6195 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977964] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977967] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977971] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977974] [ 4355] 108 4355 6792 70 0 0 0 pickup
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977977] [ 6563] 0 6563 89584 2534 0 0 0 php5-fpm
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977981] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977984] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977987] [ 7010] 1012 7010 422712 359842 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977990] [ 7672] 1012 7672 148826 85661 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977993] [ 7721] 1012 7721 339497 276600 0 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977996] [ 7918] 1012 7918 147228 83955 0 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.977999] [ 7995] 1012 7995 120170 51148 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978002] [ 8066] 1012 8066 76433 13825 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978007] [ 8071] 1012 8071 1099 25 0 0 0 sh
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978010] [ 8072] 1012 8072 68642 6036 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978013] [ 8169] 1012 8169 1099 25 0 0 0 sh
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978016] [ 8170] 1012 8170 68260 5715 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978019] [ 8176] 1012 8176 1099 25 0 0 0 sh
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978021] [ 8177] 1012 8177 68260 5716 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978024] [ 8183] 1012 8183 67117 4589 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978027] [ 8186] 1012 8186 1099 24 0 0 0 sh
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978030] [ 8188] 1012 8188 66797 4285 1 0 0 php
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978033] Out of memory: Kill process 7010 (php) score 375 or sacrifice child
Jul 16 08:51:29 ip-10-61-174-46 kernel: [1491584.978044] Killed process 7010 (php) total-vm:1690848kB, anon-rss:1439364kB, file-rss:4kB
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392851] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392855] php cpuset=/ mems_allowed=0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392858] Pid: 8183, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392859] Call Trace:
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392867] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392892] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392895] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392899] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392903] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392907] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392910] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392914] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392919] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392923] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392927] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392930] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392932] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392935] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392938] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392941] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392944] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392946] Mem-Info:
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392947] Node 0 DMA per-cpu:
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392949] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392950] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392952] Node 0 DMA32 per-cpu:
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392953] CPU 0: hi: 186, btch: 31 usd: 53
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392955] CPU 1: hi: 186, btch: 31 usd: 95
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392957] active_anon:931499 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392958] active_file:33 inactive_file:0 isolated_file:15
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392959] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392959] free:2798 slab_reclaimable:2331 slab_unreclaimable:3045
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392960] mapped:1381 shmem:1405 pagetables:4182 bounce:0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392961] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392968] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392971] Node 0 DMA32 free:2856kB min:7848kB low:9808kB high:11772kB active_anon:3725996kB inactive_anon:5572kB active_file:132kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):60kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5620kB slab_reclaimable:9324kB slab_unreclaimable:12180kB kernel_stack:888kB pagetables:16728kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:448 all_unreclaimable? yes
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392977] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392980] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392987] Node 0 DMA32: 16*4kB 15*8kB 18*16kB 5*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 2872kB
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392993] 1425 total pagecache pages
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392994] 0 pages in swap cache
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392996] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392997] Free swap = 0kB
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.392998] Total swap = 0kB
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398633] 985072 pages RAM
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398635] 24998 pages reserved
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398637] 3774 pages shared
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398638] 954439 pages non-shared
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398640] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398648] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398652] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398655] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398658] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398661] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398665] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398668] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398671] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398675] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398678] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398682] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398685] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398688] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398691] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398694] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398697] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398700] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398703] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398707] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398711] [ 3502] 1012 3502 69878 5733 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398713] [ 3504] 1012 3504 68832 6194 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398716] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398719] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398722] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398726] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398729] [ 6563] 0 6563 89584 2518 0 0 0 php5-fpm
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398732] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398735] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398739] [ 7672] 1012 7672 246774 183877 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398742] [ 7721] 1012 7721 462000 399115 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398745] [ 7918] 1012 7918 246326 183460 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398748] [ 7995] 1012 7995 137498 70304 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398751] [ 8066] 1012 8066 80329 17753 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398754] [ 8169] 1012 8169 1099 25 0 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398757] [ 8170] 1012 8170 68424 5868 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398760] [ 8176] 1012 8176 1099 25 0 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398763] [ 8177] 1012 8177 68424 5868 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398765] [ 8183] 1012 8183 69955 7368 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398768] [ 8186] 1012 8186 1099 24 0 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398771] [ 8188] 1012 8188 68424 5869 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398774] [ 8195] 1012 8195 1099 24 0 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398777] [ 8196] 1012 8196 68359 5796 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398780] [ 8202] 1012 8202 1099 26 0 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398783] [ 8203] 1012 8203 4669 58 0 0 0 git
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398786] [ 8204] 1012 8204 68476 5523 1 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398789] [ 8210] 1012 8210 1099 25 1 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398792] [ 8211] 1012 8211 4738 65 1 0 0 git
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398795] [ 8212] 1012 8212 68399 5462 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398798] [ 8218] 1012 8218 1099 25 1 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398801] [ 8219] 1012 8219 5216 67 1 0 0 git
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398803] [ 8220] 1012 8220 1099 25 0 0 0 sh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398807] [ 8221] 1012 8221 43811 622 0 0 0 php
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398809] [ 8222] 1012 8222 10901 175 0 0 0 ssh
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398812] Out of memory: Kill process 7721 (php) score 416 or sacrifice child
Jul 16 08:51:33 ip-10-61-174-46 kernel: [1491589.398823] Killed process 7721 (php) total-vm:1848000kB, anon-rss:1596460kB, file-rss:0kB
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824117] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824122] php cpuset=/ mems_allowed=0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824125] Pid: 7918, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824128] Call Trace:
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824135] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824138] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824141] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824145] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824149] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824153] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824156] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824160] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824163] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824167] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824170] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824174] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824178] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824182] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824184] Mem-Info:
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824185] Node 0 DMA per-cpu:
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824187] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824189] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824191] Node 0 DMA32 per-cpu:
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824193] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824195] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824198] active_anon:931860 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824199] active_file:15 inactive_file:4 isolated_file:0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824200] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824201] free:2834 slab_reclaimable:2321 slab_unreclaimable:3052
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824202] mapped:1352 shmem:1405 pagetables:3921 bounce:0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824204] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824212] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824216] Node 0 DMA32 free:3000kB min:7848kB low:9808kB high:11772kB active_anon:3727440kB inactive_anon:5572kB active_file:60kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9284kB slab_unreclaimable:12208kB kernel_stack:832kB pagetables:15684kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4731 all_unreclaimable? yes
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824225] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824228] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824237] Node 0 DMA32: 92*4kB 13*8kB 16*16kB 7*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3000kB
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824247] 1424 total pagecache pages
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824248] 0 pages in swap cache
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824250] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824251] Free swap = 0kB
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.824252] Total swap = 0kB
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829755] 985072 pages RAM
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829757] 24998 pages reserved
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829759] 3781 pages shared
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829760] 954562 pages non-shared
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829762] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829771] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829774] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829777] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829780] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829784] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829787] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829790] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829794] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829798] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829802] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829805] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829808] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829811] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829814] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829817] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829820] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829823] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829826] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829830] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829833] [ 3502] 1012 3502 69878 5734 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829836] [ 3504] 1012 3504 68832 6195 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829839] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829842] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829845] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829849] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829852] [ 6563] 0 6563 89584 2490 1 0 0 php5-fpm
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829855] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829858] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829861] [ 7672] 1012 7672 371253 308319 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829864] [ 7918] 1012 7918 338739 275834 0 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829867] [ 7995] 1012 7995 260663 197799 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829870] [ 8066] 1012 8066 95672 29406 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829873] [ 8183] 1012 8183 114710 45354 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829876] [ 8186] 1012 8186 1099 24 0 0 0 sh
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829879] [ 8188] 1012 8188 68842 6246 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829882] [ 8195] 1012 8195 1099 24 0 0 0 sh
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829885] [ 8196] 1012 8196 68642 6036 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829888] [ 8240] 1012 8240 79762 17139 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829891] [ 8285] 1012 8285 1099 24 0 0 0 sh
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829894] [ 8286] 1012 8286 66830 4300 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829897] [ 8300] 1012 8300 67117 4589 0 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829900] [ 8309] 1012 8309 1099 24 1 0 0 sh
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829903] [ 8310] 1012 8310 64522 2485 1 0 0 php
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829906] Out of memory: Kill process 7672 (php) score 321 or sacrifice child
Jul 16 08:51:41 ip-10-61-174-46 kernel: [1491596.829917] Killed process 7672 (php) total-vm:1485012kB, anon-rss:1233276kB, file-rss:0kB
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828155] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828160] php cpuset=/ mems_allowed=0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828164] Pid: 7995, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828166] Call Trace:
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828174] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828177] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828180] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828183] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828188] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828192] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828195] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828199] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828202] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828206] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828210] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828213] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828217] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828220] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828222] Mem-Info:
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828224] Node 0 DMA per-cpu:
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828226] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828228] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828229] Node 0 DMA32 per-cpu:
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828232] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828234] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828237] active_anon:931787 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828238] active_file:11 inactive_file:1 isolated_file:0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828239] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828240] free:2993 slab_reclaimable:2325 slab_unreclaimable:2996
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828241] mapped:1370 shmem:1405 pagetables:4052 bounce:0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828251] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828255] Node 0 DMA32 free:3636kB min:7848kB low:9808kB high:11772kB active_anon:3727148kB inactive_anon:5572kB active_file:44kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5620kB slab_reclaimable:9300kB slab_unreclaimable:11984kB kernel_stack:856kB pagetables:16208kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:314 all_unreclaimable? yes
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828264] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828267] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828277] Node 0 DMA32: 123*4kB 23*8kB 44*16kB 7*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3652kB
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828286] 1446 total pagecache pages
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828288] 0 pages in swap cache
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828289] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828291] Free swap = 0kB
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.828292] Total swap = 0kB
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835572] 985072 pages RAM
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835574] 24998 pages reserved
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835575] 3775 pages shared
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835577] 954398 pages non-shared
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835579] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835587] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835591] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835594] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835597] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835600] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835603] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835606] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835610] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835613] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835616] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835619] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835623] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835626] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835629] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835632] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835635] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835638] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835641] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835644] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835648] [ 3502] 1012 3502 69878 5734 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835651] [ 3504] 1012 3504 68832 6195 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835653] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835656] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835659] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835662] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835666] [ 6563] 0 6563 89584 2527 0 0 0 php5-fpm
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835669] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835672] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835675] [ 7918] 1012 7918 469819 406950 0 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835677] [ 7995] 1012 7995 382582 319675 0 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835681] [ 8066] 1012 8066 101930 33738 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835683] [ 8183] 1012 8183 134330 65723 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835686] [ 8240] 1012 8240 101138 32623 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835689] [ 8285] 1012 8285 1099 24 0 0 0 sh
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835692] [ 8286] 1012 8286 68424 5870 0 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835695] [ 8300] 1012 8300 68169 5594 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835698] [ 8309] 1012 8309 1099 24 1 0 0 sh
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835701] [ 8310] 1012 8310 68359 5797 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835704] [ 8317] 1012 8317 1099 25 0 0 0 sh
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835707] [ 8318] 1012 8318 68359 5797 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835710] [ 8320] 1012 8320 1099 26 0 0 0 sh
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835712] [ 8321] 1012 8321 68359 5797 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835715] [ 8323] 1012 8323 68169 5589 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835718] [ 8329] 1012 8329 1099 25 0 0 0 sh
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835721] [ 8330] 1012 8330 4776 58 1 0 0 git
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835724] [ 8331] 1012 8331 67306 4324 1 0 0 php
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835727] Out of memory: Kill process 7918 (php) score 424 or sacrifice child
Jul 16 08:51:46 ip-10-61-174-46 kernel: [1491601.835738] Killed process 7918 (php) total-vm:1879276kB, anon-rss:1627800kB, file-rss:0kB
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840132] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840137] collectd cpuset=/ mems_allowed=0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840140] Pid: 1073, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840142] Call Trace:
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840149] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840153] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840156] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840160] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840163] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840168] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840171] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840175] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840180] [<ffffffff8165e56e>] ? _raw_spin_lock_irqsave+0x2e/0x40
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840186] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840189] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840193] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840196] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840199] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840202] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840206] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840210] [<ffffffff810a152c>] ? do_futex+0x7c/0x1b0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840213] [<ffffffff810a17a2>] ? sys_futex+0x142/0x1a0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840216] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840222] Mem-Info:
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840223] Node 0 DMA per-cpu:
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840226] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840227] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840229] Node 0 DMA32 per-cpu:
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840231] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840233] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840237] active_anon:931850 inactive_anon:1393 isolated_anon:0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840238] active_file:18 inactive_file:1 isolated_file:0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840239] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840239] free:2822 slab_reclaimable:2369 slab_unreclaimable:3057
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840240] mapped:1352 shmem:1405 pagetables:3828 bounce:0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840251] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840255] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3727400kB inactive_anon:5572kB active_file:72kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5620kB slab_reclaimable:9476kB slab_unreclaimable:12228kB kernel_stack:816kB pagetables:15312kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1786 all_unreclaimable? yes
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840263] lowmem_reserve[]: 0 0 0 0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840267] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840276] Node 0 DMA32: 238*4kB 4*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2952kB
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840286] 1424 total pagecache pages
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840287] 0 pages in swap cache
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840289] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840290] Free swap = 0kB
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.840291] Total swap = 0kB
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845795] 985072 pages RAM
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845798] 24998 pages reserved
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845799] 9609 pages shared
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845800] 948744 pages non-shared
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845802] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845810] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845814] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845817] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845821] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845824] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845827] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845830] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845834] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845838] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845841] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845844] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845847] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845850] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845853] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845856] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845860] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845863] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845866] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845869] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845873] [ 3502] 1012 3502 69878 5733 1 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845876] [ 3504] 1012 3504 68832 6197 1 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845879] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845882] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845885] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845888] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845891] [ 6563] 0 6563 89584 2511 0 0 0 php5-fpm
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845895] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845898] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845901] [ 7995] 1012 7995 610884 548056 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845904] [ 8066] 1012 8066 114906 45527 1 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845907] [ 8183] 1012 8183 272145 209306 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845910] [ 8240] 1012 8240 136514 69089 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845913] [ 8300] 1012 8300 68169 5594 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845916] [ 8323] 1012 8323 68169 5594 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845919] [ 8419] 1012 8419 1099 25 0 0 0 sh
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845922] [ 8420] 1012 8420 68424 5847 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845925] [ 8436] 1012 8436 1099 25 0 0 0 sh
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845928] [ 8437] 1012 8437 67416 4866 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845930] [ 8439] 1012 8439 1099 24 0 0 0 sh
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845933] [ 8440] 1012 8440 65823 3794 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845936] [ 8442] 1012 8442 68424 5847 0 0 0 php
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845939] Out of memory: Kill process 7995 (php) score 572 or sacrifice child
Jul 16 08:51:54 ip-10-61-174-46 kernel: [1491609.845950] Killed process 7995 (php) total-vm:2443536kB, anon-rss:2192224kB, file-rss:0kB
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388347] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388351] php cpuset=/ mems_allowed=0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388354] Pid: 8066, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388356] Call Trace:
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388363] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388365] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388367] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388371] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388376] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388379] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388382] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388384] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388387] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388390] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388392] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388396] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388398] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388400] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388403] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388405] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388409] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388412] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388416] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388419] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388421] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388423] Mem-Info:
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388424] Node 0 DMA per-cpu:
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388426] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388427] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388428] Node 0 DMA32 per-cpu:
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388430] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388431] CPU 1: hi: 186, btch: 31 usd: 21
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388434] active_anon:932606 inactive_anon:1393 isolated_anon:0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388435] active_file:0 inactive_file:0 isolated_file:0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388435] unevictable:0 dirty:34 writeback:0 unstable:0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388436] free:4043 slab_reclaimable:2423 slab_unreclaimable:2736
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388437] mapped:1351 shmem:1405 pagetables:3430 bounce:0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388438] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388445] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388448] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3730424kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:136kB writeback:0kB mapped:5404kB shmem:5620kB slab_reclaimable:9692kB slab_unreclaimable:10944kB kernel_stack:760kB pagetables:13720kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:26 all_unreclaimable? yes
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388454] lowmem_reserve[]: 0 0 0 0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388457] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388464] Node 0 DMA32: 803*4kB 294*8kB 16*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388470] 1429 total pagecache pages
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388472] 0 pages in swap cache
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388473] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388474] Free swap = 0kB
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.388475] Total swap = 0kB
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.393993] 985072 pages RAM
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.393995] 24998 pages reserved
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.393996] 3797 pages shared
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.393997] 953283 pages non-shared
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.393999] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394008] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394011] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394014] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394018] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394021] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394024] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394028] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394031] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394035] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394038] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394041] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394044] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394047] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394051] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394054] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394057] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394060] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394063] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394066] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394070] [ 3502] 1012 3502 69878 5734 1 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394073] [ 3504] 1012 3504 68832 6196 1 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394076] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394079] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394082] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394086] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394089] [ 6563] 0 6563 89584 2495 1 0 0 php5-fpm
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394092] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394095] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394098] [ 8066] 1012 8066 224182 157047 0 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394101] [ 8183] 1012 8183 520894 458009 0 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394104] [ 8240] 1012 8240 329651 266729 0 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394107] [ 8443] 1012 8443 68169 5594 1 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394110] [ 8470] 1012 8470 68169 5594 1 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394113] [ 8538] 1012 8538 68169 5593 1 0 0 php
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394116] Out of memory: Kill process 8183 (php) score 478 or sacrifice child
Jul 16 08:52:02 ip-10-61-174-46 kernel: [1491618.394127] Killed process 8183 (php) total-vm:2083576kB, anon-rss:1832036kB, file-rss:0kB
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904119] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904123] ntpd cpuset=/ mems_allowed=0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904127] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904129] Call Trace:
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904137] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904143] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904146] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904152] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904156] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904160] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904163] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904167] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904170] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904174] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904177] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904181] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904185] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904189] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904193] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904196] Mem-Info:
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904197] Node 0 DMA per-cpu:
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904200] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904202] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904203] Node 0 DMA32 per-cpu:
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904205] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904207] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904211] active_anon:932834 inactive_anon:1393 isolated_anon:0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904212] active_file:24 inactive_file:0 isolated_file:0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904212] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904213] free:4047 slab_reclaimable:2279 slab_unreclaimable:2710
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904214] mapped:1357 shmem:1405 pagetables:3434 bounce:0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904216] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904225] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904229] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3731336kB inactive_anon:5572kB active_file:96kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5620kB slab_reclaimable:9116kB slab_unreclaimable:10840kB kernel_stack:760kB pagetables:13736kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1610 all_unreclaimable? yes
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904237] lowmem_reserve[]: 0 0 0 0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904241] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904250] Node 0 DMA32: 485*4kB 469*8kB 10*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7996kB
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904259] 1447 total pagecache pages
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904261] 0 pages in swap cache
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904263] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904264] Free swap = 0kB
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.904265] Total swap = 0kB
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909908] 985072 pages RAM
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909910] 24998 pages reserved
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909912] 3781 pages shared
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909913] 953319 pages non-shared
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909915] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909923] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909927] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909930] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909933] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909936] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909940] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909943] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909946] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909951] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909954] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909957] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909960] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909963] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909966] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909969] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909972] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909975] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909978] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909982] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909985] [ 3502] 1012 3502 69878 5734 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909988] [ 3504] 1012 3504 68832 6196 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909991] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909994] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.909998] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910001] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910004] [ 6563] 0 6563 89584 2529 1 0 0 php5-fpm
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910007] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910010] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910013] [ 8066] 1012 8066 455226 392330 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910017] [ 8240] 1012 8240 547008 484092 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910019] [ 8443] 1012 8443 68169 5593 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910022] [ 8470] 1012 8470 68169 5593 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910025] [ 8538] 1012 8538 68169 5593 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910028] [ 8559] 1012 8559 68169 5589 1 0 0 php
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910031] Out of memory: Kill process 8240 (php) score 505 or sacrifice child
Jul 16 08:52:11 ip-10-61-174-46 kernel: [1491626.910042] Killed process 8240 (php) total-vm:2188032kB, anon-rss:1936368kB, file-rss:0kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044239] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044244] collectd cpuset=/ mems_allowed=0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044248] Pid: 1074, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044250] Call Trace:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044258] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044261] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044264] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044268] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044272] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044276] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044279] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044283] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044288] [<ffffffff813f3bc7>] ? put_device+0x17/0x20
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044292] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044297] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044300] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044303] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044308] [<ffffffff81319ce6>] ? copy_user_enhanced_fast_string+0x6/0x10
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044312] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044316] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044320] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044323] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044325] Mem-Info:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044327] Node 0 DMA per-cpu:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044330] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044333] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044335] Node 0 DMA32 per-cpu:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044337] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044339] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044342] active_anon:932899 inactive_anon:1393 isolated_anon:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044343] active_file:0 inactive_file:9 isolated_file:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044344] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044345] free:4048 slab_reclaimable:2208 slab_unreclaimable:2650
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044346] mapped:1378 shmem:1405 pagetables:3447 bounce:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044348] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044356] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044360] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731596kB inactive_anon:5572kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5512kB shmem:5620kB slab_reclaimable:8832kB slab_unreclaimable:10600kB kernel_stack:776kB pagetables:13788kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044369] lowmem_reserve[]: 0 0 0 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044374] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044383] Node 0 DMA32: 274*4kB 610*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8040kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044393] 1458 total pagecache pages
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044394] 0 pages in swap cache
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044396] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044398] Free swap = 0kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.044399] Total swap = 0kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051713] 985072 pages RAM
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051715] 24998 pages reserved
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051716] 3801 pages shared
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051717] 953288 pages non-shared
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051719] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051728] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051732] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051735] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051738] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051741] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051745] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051748] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051751] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051755] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051758] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051761] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051766] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051769] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051772] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051775] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051779] [ 1051] 0 1051 201626 844 0 0 0 collectd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051782] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051785] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051788] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051792] [ 3502] 1012 3502 69878 5734 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051795] [ 3504] 1012 3504 68832 6199 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051798] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051801] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051804] [ 3533] 1012 3533 68271 5620 0 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051808] [ 4355] 108 4355 6792 71 0 0 0 pickup
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051811] [ 6563] 0 6563 89584 2514 0 0 0 php5-fpm
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051814] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051818] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051821] [ 8066] 1012 8066 939480 876627 0 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051824] [ 8443] 1012 8443 68169 5593 0 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051827] [ 8559] 1012 8559 68169 5593 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051830] [ 8597] 1012 8597 68169 5593 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051833] [ 8626] 1012 8626 1099 25 1 0 0 sh
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051836] [ 8627] 1012 8627 64782 2721 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051838] [ 8628] 1012 8628 1099 24 1 0 0 sh
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051841] [ 8629] 1012 8629 64782 2721 1 0 0 php
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051844] Out of memory: Kill process 8066 (php) score 914 or sacrifice child
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.051858] Killed process 8066 (php) total-vm:3757920kB, anon-rss:3506508kB, file-rss:0kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076094] php: page allocation failure: order:0, mode:0x201da
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076098] Pid: 8066, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076100] Call Trace:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076107] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076114] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076119] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076123] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076128] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076131] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076136] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076139] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076143] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076146] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076149] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076155] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076158] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076161] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076165] [<ffffffff811425e0>] ? find_vma+0x30/0x70
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076168] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076172] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076174] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076178] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076180] Mem-Info:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076182] Node 0 DMA per-cpu:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076184] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076186] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076187] Node 0 DMA32 per-cpu:
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076189] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076191] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076196] active_anon:932899 inactive_anon:1393 isolated_anon:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076197] active_file:0 inactive_file:9 isolated_file:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076198] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076199] free:4048 slab_reclaimable:2208 slab_unreclaimable:2650
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076200] mapped:1378 shmem:1405 pagetables:3447 bounce:0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076202] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076210] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076214] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3731596kB inactive_anon:5572kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5512kB shmem:5620kB slab_reclaimable:8832kB slab_unreclaimable:10600kB kernel_stack:776kB pagetables:13788kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076222] lowmem_reserve[]: 0 0 0 0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076225] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076237] Node 0 DMA32: 274*4kB 610*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8040kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076246] 1458 total pagecache pages
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076247] 0 pages in swap cache
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076249] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076251] Free swap = 0kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.076252] Total swap = 0kB
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.083296] 985072 pages RAM
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.083299] 24998 pages reserved
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.083300] 3801 pages shared
Jul 16 08:52:24 ip-10-61-174-46 kernel: [1491639.083301] 953288 pages non-shared
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351640] ssh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351645] ssh cpuset=/ mems_allowed=0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351648] Pid: 24298, comm: ssh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351650] Call Trace:
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351657] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351660] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351662] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351665] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351668] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351672] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351674] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351677] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351681] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351683] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351687] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351689] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351691] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351694] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351697] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351700] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351703] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351705] Mem-Info:
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351707] Node 0 DMA per-cpu:
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351709] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351710] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351711] Node 0 DMA32 per-cpu:
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351713] CPU 0: hi: 186, btch: 31 usd: 92
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351714] CPU 1: hi: 186, btch: 31 usd: 80
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351717] active_anon:930119 inactive_anon:1396 isolated_anon:0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351717] active_file:0 inactive_file:23 isolated_file:21
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351718] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351719] free:4039 slab_reclaimable:3115 slab_unreclaimable:3542
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351720] mapped:1381 shmem:1408 pagetables:4082 bounce:0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351721] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351728] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351731] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3720476kB inactive_anon:5584kB active_file:0kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):84kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5524kB shmem:5632kB slab_reclaimable:12460kB slab_unreclaimable:14168kB kernel_stack:928kB pagetables:16328kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:260 all_unreclaimable? yes
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351737] lowmem_reserve[]: 0 0 0 0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351740] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351746] Node 0 DMA32: 1437*4kB 3*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351753] 1476 total pagecache pages
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351754] 0 pages in swap cache
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351755] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351757] Free swap = 0kB
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.351757] Total swap = 0kB
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357479] 985072 pages RAM
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357481] 24998 pages reserved
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357483] 4257 pages shared
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357484] 952705 pages non-shared
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357486] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357494] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357498] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357501] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357504] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357508] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357511] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357514] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357518] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357522] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357525] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357528] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357531] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357535] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357539] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357545] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357548] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357551] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357555] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357558] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357561] [ 3502] 1012 3502 71863 7727 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357564] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357567] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357571] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357576] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357579] [ 6563] 0 6563 89584 2541 1 0 0 php5-fpm
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357583] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357587] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357591] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357594] [11050] 1002 11050 18359 198 0 0 0 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357597] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357600] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357603] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357606] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357609] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357612] [25434] 1002 25434 18741 577 0 0 0 sshd
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357615] [25450] 1002 25450 6542 1749 0 0 0 bash
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357618] [26629] 1002 26629 396594 333645 0 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357621] [29474] 108 29474 6792 70 1 0 0 pickup
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357625] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357628] [19896] 108 19896 6818 73 1 0 0 cleanup
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357631] [19897] 108 19897 6795 70 0 0 0 trivial-rewrite
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357634] [21040] 108 21040 11107 158 0 0 0 smtp
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357637] [21069] 108 21069 11107 157 0 0 0 smtp
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357641] [24140] 1012 24140 380539 317675 0 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357644] [24222] 1012 24222 71693 9127 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357647] [24224] 1012 24224 208053 125945 0 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357650] [24228] 1012 24228 71713 9122 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357653] [24280] 1012 24280 1099 24 0 0 0 sh
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357656] [24281] 1012 24281 68424 5868 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357659] [24291] 1012 24291 1099 26 1 0 0 sh
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357662] [24292] 1012 24292 4724 58 0 0 0 git
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357665] [24293] 1012 24293 68476 5522 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357668] [24296] 1012 24296 1099 25 0 0 0 sh
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357671] [24298] 1012 24298 10901 175 1 0 0 ssh
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357674] [24303] 1012 24303 11724 99 1 0 0 php
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357677] Out of memory: Kill process 26629 (php) score 348 or sacrifice child
Jul 16 10:56:16 ip-10-61-174-46 kernel: [1499072.357688] Killed process 26629 (php) total-vm:1586376kB, anon-rss:1334580kB, file-rss:0kB
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892173] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892177] php cpuset=/ mems_allowed=0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892180] Pid: 24224, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892182] Call Trace:
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892190] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892193] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892196] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892200] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892206] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892209] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892214] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892217] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892221] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892225] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892229] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892232] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892236] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892239] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892243] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892246] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892249] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892252] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892256] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892259] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892262] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892266] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892268] Mem-Info:
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892269] Node 0 DMA per-cpu:
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892272] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892274] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892275] Node 0 DMA32 per-cpu:
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892278] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892280] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892284] active_anon:930636 inactive_anon:1396 isolated_anon:0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892285] active_file:0 inactive_file:34 isolated_file:0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892286] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892286] free:4048 slab_reclaimable:2550 slab_unreclaimable:3486
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892287] mapped:1355 shmem:1408 pagetables:4278 bounce:0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892289] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892297] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892301] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3722544kB inactive_anon:5584kB active_file:0kB inactive_file:136kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:10200kB slab_unreclaimable:13944kB kernel_stack:960kB pagetables:17112kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14552 all_unreclaimable? yes
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892310] lowmem_reserve[]: 0 0 0 0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892313] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892323] Node 0 DMA32: 1142*4kB 163*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892332] 1438 total pagecache pages
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892333] 0 pages in swap cache
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892335] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892336] Free swap = 0kB
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.892337] Total swap = 0kB
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898554] 985072 pages RAM
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898556] 24998 pages reserved
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898557] 4244 pages shared
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898559] 952833 pages non-shared
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898561] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898568] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898572] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898575] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898578] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898581] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898585] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898588] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898591] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898594] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898597] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898600] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898604] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898607] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898610] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898613] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898616] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898618] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898622] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898625] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898628] [ 3502] 1012 3502 71863 7729 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898631] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898634] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898636] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898640] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898642] [ 6563] 0 6563 89584 2509 0 0 0 php5-fpm
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898645] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898648] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898652] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898655] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898658] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898661] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898664] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898666] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898669] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898672] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898675] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898679] [29474] 108 29474 6792 70 1 0 0 pickup
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898682] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898685] [19897] 108 19897 6795 70 0 0 0 trivial-rewrite
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898688] [24140] 1012 24140 524100 461274 0 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898691] [24224] 1012 24224 354228 291365 0 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898694] [24303] 1012 24303 73105 10518 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898696] [24386] 1012 24386 72999 10424 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898699] [24390] 1012 24390 1099 24 1 0 0 sh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898703] [24393] 1012 24393 68424 5866 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898706] [24394] 1012 24394 1099 24 0 0 0 sh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898709] [24395] 1012 24395 68424 5869 0 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898711] [24406] 1012 24406 70310 7714 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898714] [24412] 1012 24412 1099 25 0 0 0 sh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898718] [24413] 1012 24413 4769 91 0 0 0 git
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898721] [24414] 1012 24414 68476 5523 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898724] [24420] 1012 24420 1099 26 0 0 0 sh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898726] [24421] 1012 24421 14084 63 0 0 0 git
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898729] [24422] 1012 24422 68476 5523 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898732] [24424] 1012 24424 1099 25 1 0 0 sh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898735] [24425] 1012 24425 10901 166 1 0 0 ssh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898738] [24428] 1012 24428 65639 3553 1 0 0 php
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898741] [24433] 1012 24433 5352 75 1 0 0 git
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898744] [24434] 1012 24434 1099 25 1 0 0 sh
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898746] Out of memory: Kill process 24140 (php) score 481 or sacrifice child
Jul 16 10:56:24 ip-10-61-174-46 kernel: [1499079.898758] Killed process 24140 (php) total-vm:2096400kB, anon-rss:1845096kB, file-rss:0kB
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812130] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812135] php cpuset=/ mems_allowed=0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812138] Pid: 24439, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812140] Call Trace:
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812148] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812151] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812154] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812158] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812162] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812166] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812169] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812173] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812176] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812180] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812184] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812187] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812191] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812195] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812198] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812202] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812205] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812209] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812212] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812214] Mem-Info:
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812216] Node 0 DMA per-cpu:
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812218] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812220] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812222] Node 0 DMA32 per-cpu:
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812224] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812226] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812230] active_anon:930731 inactive_anon:1396 isolated_anon:0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812230] active_file:16 inactive_file:2 isolated_file:0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812231] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812232] free:2990 slab_reclaimable:2330 slab_unreclaimable:3786
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812233] mapped:1355 shmem:1408 pagetables:3830 bounce:0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812244] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812248] Node 0 DMA32 free:3624kB min:7848kB low:9808kB high:11772kB active_anon:3722924kB inactive_anon:5584kB active_file:64kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9320kB slab_unreclaimable:15144kB kernel_stack:872kB pagetables:15320kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1840 all_unreclaimable? yes
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812256] lowmem_reserve[]: 0 0 0 0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812260] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812269] Node 0 DMA32: 412*4kB 17*8kB 19*16kB 0*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3624kB
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812279] 1426 total pagecache pages
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812280] 0 pages in swap cache
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812282] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812283] Free swap = 0kB
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.812285] Total swap = 0kB
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819521] 985072 pages RAM
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819523] 24998 pages reserved
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819525] 4227 pages shared
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819526] 953951 pages non-shared
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819528] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819536] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819539] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819542] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819545] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819548] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819552] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819555] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819559] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819562] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819566] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819569] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819572] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819575] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819578] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819581] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819584] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819588] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819591] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819594] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819597] [ 3502] 1012 3502 71863 7732 0 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819600] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819603] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819606] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819609] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819612] [ 6563] 0 6563 89584 2493 1 0 0 php5-fpm
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819615] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819619] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819622] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819625] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819628] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819631] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819634] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819637] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819640] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819642] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819645] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819649] [29474] 108 29474 6792 70 1 0 0 pickup
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819652] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819655] [24224] 1012 24224 569921 507056 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819658] [24428] 1012 24428 294895 231971 0 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819661] [24439] 1012 24439 73113 10583 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819664] [24572] 1012 24572 116351 47120 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819667] [24660] 1012 24660 1099 25 0 0 0 sh
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819670] [24661] 1012 24661 68424 5869 0 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819673] [24683] 1012 24683 1099 26 1 0 0 sh
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819676] [24684] 1012 24684 4651 57 1 0 0 git
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819679] [24685] 1012 24685 68476 5523 1 0 0 php
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819682] [24689] 1012 24689 1099 25 0 0 0 sh
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819685] Out of memory: Kill process 24224 (php) score 529 or sacrifice child
Jul 16 10:56:36 ip-10-61-174-46 kernel: [1499091.819697] Killed process 24224 (php) total-vm:2279684kB, anon-rss:2028224kB, file-rss:0kB
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.115992] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.115997] ntpd cpuset=/ mems_allowed=0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116000] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116002] Call Trace:
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116011] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116014] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116017] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116021] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116026] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116031] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116034] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116037] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116041] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116046] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116049] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116068] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116072] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116076] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116081] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116083] Mem-Info:
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116085] Node 0 DMA per-cpu:
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116087] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116089] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116090] Node 0 DMA32 per-cpu:
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116093] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116095] CPU 1: hi: 186, btch: 31 usd: 115
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116098] active_anon:931044 inactive_anon:1396 isolated_anon:0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116099] active_file:0 inactive_file:14 isolated_file:19
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116100] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116101] free:4043 slab_reclaimable:2414 slab_unreclaimable:3330
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116102] mapped:1378 shmem:1408 pagetables:4089 bounce:0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116104] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116113] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116117] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3724176kB inactive_anon:5584kB active_file:0kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5512kB shmem:5632kB slab_reclaimable:9656kB slab_unreclaimable:13320kB kernel_stack:904kB pagetables:16356kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:207 all_unreclaimable? yes
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116125] lowmem_reserve[]: 0 0 0 0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116129] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116139] Node 0 DMA32: 1447*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116149] 1409 total pagecache pages
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116151] 0 pages in swap cache
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116152] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116154] Free swap = 0kB
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.116155] Total swap = 0kB
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121841] 985072 pages RAM
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121843] 24998 pages reserved
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121844] 4233 pages shared
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121845] 952736 pages non-shared
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121846] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121854] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121857] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121859] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121861] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121863] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121866] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121868] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121871] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121874] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121877] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121879] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121881] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121883] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121885] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121888] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121890] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121892] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121894] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121896] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121899] [ 3502] 1012 3502 71863 7734 1 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121901] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121903] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121905] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121908] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121910] [ 6563] 0 6563 89584 2527 0 0 0 php5-fpm
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121912] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121914] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121917] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121919] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121921] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121923] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121926] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121928] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121930] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121932] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121934] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121936] [29474] 108 29474 6792 70 0 0 0 pickup
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121939] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121941] [24428] 1012 24428 473657 410800 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121943] [24572] 1012 24572 315256 252407 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121946] [24692] 1012 24692 163550 99043 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121948] [24723] 1012 24723 72911 10352 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121950] [24726] 1012 24726 72976 10432 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121952] [24751] 1012 24751 1099 25 0 0 0 sh
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121954] [24752] 1012 24752 68424 5871 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121956] [24754] 1012 24754 71962 9362 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121958] [24762] 1012 24762 1099 25 1 0 0 sh
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121960] [24763] 1012 24763 14084 62 1 0 0 git
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121962] [24764] 1012 24764 68476 5523 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121964] [24770] 1012 24770 66597 4543 0 0 0 php
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121966] [24772] 1012 24772 1099 25 0 0 0 sh
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121968] [24773] 1012 24773 10901 176 1 0 0 ssh
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121970] Out of memory: Kill process 24428 (php) score 428 or sacrifice child
Jul 16 10:56:44 ip-10-61-174-46 kernel: [1499100.121980] Killed process 24428 (php) total-vm:1894628kB, anon-rss:1643200kB, file-rss:0kB
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741566] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741570] php cpuset=/ mems_allowed=0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741573] Pid: 5438, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741575] Call Trace:
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741582] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741584] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741586] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741589] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741593] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741596] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741598] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741601] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741605] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741608] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741611] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741613] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741616] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741618] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741621] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741625] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741628] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741629] Mem-Info:
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741631] Node 0 DMA per-cpu:
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741633] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741634] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741635] Node 0 DMA32 per-cpu:
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741637] CPU 0: hi: 186, btch: 31 usd: 105
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741638] CPU 1: hi: 186, btch: 31 usd: 41
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741641] active_anon:930284 inactive_anon:1396 isolated_anon:0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741641] active_file:25 inactive_file:27 isolated_file:33
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741642] unevictable:0 dirty:24 writeback:0 unstable:0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741643] free:3165 slab_reclaimable:2629 slab_unreclaimable:3696
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741644] mapped:1371 shmem:1408 pagetables:3892 bounce:0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741645] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741652] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741655] Node 0 DMA32 free:4324kB min:7848kB low:9808kB high:11772kB active_anon:3721136kB inactive_anon:5584kB active_file:100kB inactive_file:108kB unevictable:0kB isolated(anon):0kB isolated(file):132kB present:3862656kB mlocked:0kB dirty:96kB writeback:0kB mapped:5484kB shmem:5632kB slab_reclaimable:10516kB slab_unreclaimable:14784kB kernel_stack:896kB pagetables:15568kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:319 all_unreclaimable? yes
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741661] lowmem_reserve[]: 0 0 0 0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741664] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741670] Node 0 DMA32: 759*4kB 25*8kB 0*16kB 0*32kB 1*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 4324kB
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741677] 1450 total pagecache pages
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741678] 0 pages in swap cache
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741679] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741681] Free swap = 0kB
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.741681] Total swap = 0kB
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747355] 985072 pages RAM
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747358] 24998 pages reserved
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747359] 4237 pages shared
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747360] 953620 pages non-shared
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747362] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747371] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747375] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747378] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747381] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747384] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747388] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747391] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747395] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747399] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747402] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747405] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747408] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747411] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747414] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747418] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747421] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747424] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747427] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747430] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747434] [ 3502] 1012 3502 71863 7735 1 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747437] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747440] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747443] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747446] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747449] [ 6563] 0 6563 89584 2511 0 0 0 php5-fpm
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747452] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747456] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747459] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747462] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747465] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747468] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747471] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747474] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747477] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747480] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747483] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747486] [29474] 108 29474 6792 70 0 0 0 pickup
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747489] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747493] [24572] 1012 24572 469313 406418 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747496] [24692] 1012 24692 364795 301917 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747499] [24770] 1012 24770 73040 10458 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747502] [24781] 1012 24781 139361 71777 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747505] [24872] 1012 24872 1099 24 0 0 0 sh
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747508] [24873] 1012 24873 68424 5890 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747511] [24892] 1012 24892 68169 5589 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747514] [24894] 1012 24894 68249 5673 0 0 0 php
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747517] [24903] 1012 24903 1099 27 0 0 0 sh
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747520] [24904] 1012 24904 4682 61 0 0 0 git
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747523] Out of memory: Kill process 24572 (php) score 424 or sacrifice child
Jul 16 10:56:52 ip-10-61-174-46 kernel: [1499107.747533] Killed process 24572 (php) total-vm:1877252kB, anon-rss:1625672kB, file-rss:0kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252126] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252130] php cpuset=/ mems_allowed=0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252133] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252135] Call Trace:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252141] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252144] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252146] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252149] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252153] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252156] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252158] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252161] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252164] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252167] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252169] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252172] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252175] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252177] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252180] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252184] [<ffffffff8106bc3c>] ? do_wait+0x12c/0x260
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252186] [<ffffffff8106cf4b>] ? sys_wait4+0xab/0xf0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252189] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252192] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252194] Mem-Info:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252195] Node 0 DMA per-cpu:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252197] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252198] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252199] Node 0 DMA32 per-cpu:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252201] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252202] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252205] active_anon:931529 inactive_anon:1396 isolated_anon:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252205] active_file:9 inactive_file:9 isolated_file:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252206] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252207] free:4048 slab_reclaimable:2481 slab_unreclaimable:3231
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252208] mapped:1362 shmem:1408 pagetables:3824 bounce:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252209] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252216] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252219] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3726116kB inactive_anon:5584kB active_file:36kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5632kB slab_reclaimable:9924kB slab_unreclaimable:12924kB kernel_stack:856kB pagetables:15296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8033 all_unreclaimable? yes
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252225] lowmem_reserve[]: 0 0 0 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252228] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252234] Node 0 DMA32: 1466*4kB 9*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8016kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252241] 1482 total pagecache pages
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252242] 0 pages in swap cache
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252243] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252244] Free swap = 0kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.252245] Total swap = 0kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257659] 985072 pages RAM
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257662] 24998 pages reserved
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257663] 4244 pages shared
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257664] 952838 pages non-shared
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257666] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257675] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257678] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257681] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257684] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257687] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257691] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257694] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257698] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257702] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257705] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257708] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257711] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257714] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257718] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257721] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257724] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257727] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257730] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257733] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257737] [ 3502] 1012 3502 71863 7738 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257739] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257743] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257745] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257749] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257752] [ 6563] 0 6563 89584 2520 0 0 0 php5-fpm
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257755] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257758] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257761] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257764] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257767] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257770] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257773] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257777] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257780] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257783] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257786] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257789] [29474] 108 29474 6792 70 0 0 0 pickup
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257792] [ 5438] 1002 5438 143346 79915 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257796] [24692] 1012 24692 536197 473317 0 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257799] [24781] 1012 24781 367542 304670 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257802] [24910] 1012 24910 72143 9584 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257805] [24954] 1012 24954 72013 9470 0 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257808] [24956] 1012 24956 1099 25 1 0 0 sh
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257811] [24957] 1012 24957 34164 1426 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257814] [24997] 1012 24997 67913 5339 0 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257817] [25003] 1012 25003 67912 5319 1 0 0 php
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257820] Out of memory: Kill process 24692 (php) score 494 or sacrifice child
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.257831] Killed process 24692 (php) total-vm:2144788kB, anon-rss:1893268kB, file-rss:0kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316081] php: page allocation failure: order:0, mode:0x201da
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316085] Pid: 24692, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316087] Call Trace:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316096] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316100] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316107] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316111] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316117] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316120] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316125] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316128] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316131] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316136] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316140] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316144] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316147] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316152] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316155] [<ffffffff811425e0>] ? find_vma+0x30/0x70
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316159] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316161] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316164] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316167] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316170] Mem-Info:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316171] Node 0 DMA per-cpu:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316174] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316176] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316177] Node 0 DMA32 per-cpu:
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316179] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316181] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316185] active_anon:931529 inactive_anon:1396 isolated_anon:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316186] active_file:9 inactive_file:16 isolated_file:2
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316187] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316188] free:4048 slab_reclaimable:2481 slab_unreclaimable:3231
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316189] mapped:1362 shmem:1408 pagetables:3824 bounce:0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316191] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316201] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316205] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3726116kB inactive_anon:5584kB active_file:36kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):8kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5632kB slab_reclaimable:9924kB slab_unreclaimable:12924kB kernel_stack:856kB pagetables:15296kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1659 all_unreclaimable? no
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316213] lowmem_reserve[]: 0 0 0 0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316217] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316226] Node 0 DMA32: 1456*4kB 10*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316236] 1445 total pagecache pages
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316238] 0 pages in swap cache
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316239] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316241] Free swap = 0kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.316242] Total swap = 0kB
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.323421] 985072 pages RAM
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.323423] 24998 pages reserved
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.323424] 4254 pages shared
Jul 16 10:57:01 ip-10-61-174-46 kernel: [1499117.323425] 952839 pages non-shared
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467120] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467124] php cpuset=/ mems_allowed=0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467127] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467128] Call Trace:
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467136] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467138] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467140] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467143] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467148] [<ffffffff81312aa6>] ? radix_tree_next_hole+0x36/0x50
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467151] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467155] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467159] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467161] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467164] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467166] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467170] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467172] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467174] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467177] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467180] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467184] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467186] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467189] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467190] Mem-Info:
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467192] Node 0 DMA per-cpu:
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467194] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467195] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467196] Node 0 DMA32 per-cpu:
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467198] CPU 0: hi: 186, btch: 31 usd: 87
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467199] CPU 1: hi: 186, btch: 31 usd: 22
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467202] active_anon:929837 inactive_anon:1396 isolated_anon:24
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467203] active_file:2 inactive_file:8 isolated_file:18
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467203] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467204] free:3564 slab_reclaimable:2487 slab_unreclaimable:3613
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467205] mapped:1374 shmem:1408 pagetables:3920 bounce:0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467206] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467213] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467216] Node 0 DMA32 free:5920kB min:7848kB low:9808kB high:11772kB active_anon:3719348kB inactive_anon:5584kB active_file:8kB inactive_file:32kB unevictable:0kB isolated(anon):96kB isolated(file):72kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5632kB slab_reclaimable:9948kB slab_unreclaimable:14452kB kernel_stack:848kB pagetables:15680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:77 all_unreclaimable? yes
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467222] lowmem_reserve[]: 0 0 0 0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467225] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467232] Node 0 DMA32: 1196*4kB 28*8kB 1*16kB 0*32kB 1*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 5984kB
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467238] 1449 total pagecache pages
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467240] 0 pages in swap cache
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467241] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467242] Free swap = 0kB
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.467243] Total swap = 0kB
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472939] 985072 pages RAM
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472941] 24998 pages reserved
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472942] 4258 pages shared
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472943] 953235 pages non-shared
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472945] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472954] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472958] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472961] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472964] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472967] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472970] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472973] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472977] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472981] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472984] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472988] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472991] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472994] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.472997] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473000] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473003] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473006] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473010] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473013] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473016] [ 3502] 1012 3502 71863 7742 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473019] [ 3504] 1012 3504 68832 6227 0 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473022] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473025] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473030] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473033] [ 6563] 0 6563 89584 2504 0 0 0 php5-fpm
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473036] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473039] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473043] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473046] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473049] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473052] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473055] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473058] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473061] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473064] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473067] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473071] [29474] 108 29474 6792 70 0 0 0 pickup
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473074] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473078] [24781] 1012 24781 741517 678665 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473081] [24954] 1012 24954 73787 11338 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473084] [25132] 1012 25132 148559 85523 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473087] [25135] 1012 25135 70981 8426 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473090] [25184] 1012 25184 70171 7628 0 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473093] [25199] 1012 25199 70250 7657 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473096] [25206] 1012 25206 70250 7702 1 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473099] [25222] 1012 25222 1099 25 0 0 0 sh
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473102] [25223] 1012 25223 31064 410 0 0 0 php
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473105] Out of memory: Kill process 24781 (php) score 708 or sacrifice child
Jul 16 10:57:14 ip-10-61-174-46 kernel: [1499129.473116] Killed process 24781 (php) total-vm:2966068kB, anon-rss:2714652kB, file-rss:8kB
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412155] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412160] php cpuset=/ mems_allowed=0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412163] Pid: 25323, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412166] Call Trace:
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412174] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412177] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412180] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412184] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412188] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412192] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412195] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412199] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412203] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412205] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412208] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412211] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412213] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412217] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412219] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412221] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412224] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412227] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412228] Mem-Info:
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412230] Node 0 DMA per-cpu:
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412232] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412233] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412234] Node 0 DMA32 per-cpu:
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412236] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412237] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412239] active_anon:931608 inactive_anon:1396 isolated_anon:0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412240] active_file:0 inactive_file:0 isolated_file:0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412241] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412242] free:4048 slab_reclaimable:2446 slab_unreclaimable:3074
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412242] mapped:1393 shmem:1408 pagetables:3883 bounce:0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412244] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412250] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412254] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3726432kB inactive_anon:5584kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5572kB shmem:5632kB slab_reclaimable:9784kB slab_unreclaimable:12296kB kernel_stack:872kB pagetables:15532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412260] lowmem_reserve[]: 0 0 0 0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412262] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412269] Node 0 DMA32: 1487*4kB 3*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8036kB
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412276] 1465 total pagecache pages
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412277] 0 pages in swap cache
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412278] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412279] Free swap = 0kB
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.412280] Total swap = 0kB
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417769] 985072 pages RAM
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417772] 24998 pages reserved
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417773] 4254 pages shared
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417775] 952816 pages non-shared
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417777] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417784] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417788] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417791] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417794] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417797] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417800] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417804] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417807] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417811] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417815] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417818] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417821] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417824] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417827] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417830] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417833] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417836] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417839] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417843] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417846] [ 3502] 1012 3502 71863 7748 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417849] [ 3504] 1012 3504 68832 6227 1 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417852] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417855] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417858] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417861] [ 6563] 0 6563 89584 2519 0 0 0 php5-fpm
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417864] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417867] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417870] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417873] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417876] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417879] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417882] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417885] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417888] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417891] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417894] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417898] [29474] 108 29474 6792 70 0 0 0 pickup
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417901] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417904] [25132] 1012 25132 665095 602250 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417907] [25308] 1012 25308 73155 10640 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417910] [25323] 1012 25323 238263 175215 1 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417912] [25386] 1012 25386 73745 11269 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417915] [25421] 1012 25421 1099 24 0 0 0 sh
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417918] [25422] 1012 25422 65497 3446 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417921] [25423] 1012 25423 65573 3511 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417924] [25424] 1012 25424 1099 26 0 0 0 sh
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417927] [25425] 1012 25425 64912 2864 0 0 0 php
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417930] Out of memory: Kill process 25132 (php) score 628 or sacrifice child
Jul 16 10:57:28 ip-10-61-174-46 kernel: [1499144.417941] Killed process 25132 (php) total-vm:2660380kB, anon-rss:2409000kB, file-rss:0kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828200] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828205] php cpuset=/ mems_allowed=0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828208] Pid: 25761, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828210] Call Trace:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828218] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828221] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828224] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828228] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828232] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828236] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828239] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828243] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828246] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828250] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828253] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828256] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828260] [<ffffffff81255faf>] ? ext4_get_acl+0x19f/0x230
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828264] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828268] [<ffffffff81185026>] ? acl_permission_check+0x86/0xb0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828271] [<ffffffff810e090d>] ? __call_rcu+0x9d/0x170
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828276] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828298] [<ffffffff810e0a15>] ? call_rcu_sched+0x15/0x20
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828302] [<ffffffff81091dff>] ? __put_cred+0x3f/0x50
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828305] [<ffffffff811783d4>] ? sys_faccessat+0x114/0x1e0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828309] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828311] Mem-Info:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828312] Node 0 DMA per-cpu:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828314] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828316] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828318] Node 0 DMA32 per-cpu:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828320] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828322] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828325] active_anon:931629 inactive_anon:1396 isolated_anon:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828325] active_file:18 inactive_file:0 isolated_file:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828326] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828327] free:2972 slab_reclaimable:2448 slab_unreclaimable:3328
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828328] mapped:1373 shmem:1408 pagetables:3605 bounce:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828330] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828338] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828342] Node 0 DMA32 free:3552kB min:7848kB low:9808kB high:11772kB active_anon:3726516kB inactive_anon:5584kB active_file:72kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5492kB shmem:5632kB slab_reclaimable:9792kB slab_unreclaimable:13312kB kernel_stack:824kB pagetables:14420kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828350] lowmem_reserve[]: 0 0 0 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828353] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828363] Node 0 DMA32: 436*4kB 26*8kB 0*16kB 0*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3552kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828372] 1451 total pagecache pages
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828374] 0 pages in swap cache
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828376] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828377] Free swap = 0kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.828378] Total swap = 0kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835648] 985072 pages RAM
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835650] 24998 pages reserved
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835651] 4236 pages shared
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835653] 953967 pages non-shared
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835655] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835663] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835666] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835670] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835673] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835675] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835678] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835681] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835685] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835688] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835691] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835695] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835698] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835701] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835704] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835707] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835709] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835712] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835715] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835718] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835722] [ 3502] 1012 3502 71927 7757 0 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835724] [ 3504] 1012 3504 68832 6227 0 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835727] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835730] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835736] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835739] [ 6563] 0 6563 89584 2537 0 0 0 php5-fpm
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835742] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835745] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835748] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835751] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835754] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835757] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835761] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835764] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835767] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835770] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835773] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835776] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835779] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835783] [25323] 1012 25323 839954 777068 1 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835786] [25761] 1012 25761 73787 11330 1 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835789] [25796] 1012 25796 76120 13578 0 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835792] [25802] 1012 25802 69732 7191 0 0 0 php
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835795] Out of memory: Kill process 25323 (php) score 811 or sacrifice child
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835808] Killed process 25323 (php) total-vm:3359816kB, anon-rss:3108272kB, file-rss:0kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835845] php: page allocation failure: order:0, mode:0x280da
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835848] Pid: 25323, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835850] Call Trace:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835859] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835865] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835870] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835873] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835878] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835881] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835884] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835888] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835891] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835894] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835899] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835903] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835906] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835911] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835917] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835921] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835923] Mem-Info:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835924] Node 0 DMA per-cpu:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835926] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835928] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835930] Node 0 DMA32 per-cpu:
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835932] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835934] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835937] active_anon:931629 inactive_anon:1396 isolated_anon:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835938] active_file:18 inactive_file:0 isolated_file:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835939] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835940] free:2972 slab_reclaimable:2448 slab_unreclaimable:3328
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835941] mapped:1373 shmem:1408 pagetables:3605 bounce:0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835943] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835952] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835956] Node 0 DMA32 free:3552kB min:7848kB low:9808kB high:11772kB active_anon:3726516kB inactive_anon:5584kB active_file:72kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5492kB shmem:5632kB slab_reclaimable:9792kB slab_unreclaimable:13312kB kernel_stack:824kB pagetables:14420kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:57 all_unreclaimable? no
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835965] lowmem_reserve[]: 0 0 0 0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835969] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835979] Node 0 DMA32: 447*4kB 26*8kB 0*16kB 0*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3596kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835988] 1451 total pagecache pages
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835990] 0 pages in swap cache
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835992] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835993] Free swap = 0kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.835994] Total swap = 0kB
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.843144] 985072 pages RAM
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.843146] 24998 pages reserved
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.843147] 4236 pages shared
Jul 16 10:57:48 ip-10-61-174-46 kernel: [1499163.843148] 953956 pages non-shared
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089707] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089711] php cpuset=/ mems_allowed=0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089714] Pid: 3502, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089716] Call Trace:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089723] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089726] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089728] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089731] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089734] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089737] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089740] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089743] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089746] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089748] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089751] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089754] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089756] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089759] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089762] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089766] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089768] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089771] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089774] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089776] Mem-Info:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089777] Node 0 DMA per-cpu:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089779] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089780] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089781] Node 0 DMA32 per-cpu:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089783] CPU 0: hi: 186, btch: 31 usd: 62
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089784] CPU 1: hi: 186, btch: 31 usd: 58
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089787] active_anon:930676 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089788] active_file:0 inactive_file:21 isolated_file:24
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089788] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089789] free:3046 slab_reclaimable:2536 slab_unreclaimable:3414
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089790] mapped:1359 shmem:1408 pagetables:3854 bounce:0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089791] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089798] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089801] Node 0 DMA32 free:3848kB min:7848kB low:9808kB high:11772kB active_anon:3722704kB inactive_anon:5584kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):96kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5632kB slab_reclaimable:10144kB slab_unreclaimable:13656kB kernel_stack:872kB pagetables:15416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:157 all_unreclaimable? yes
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089807] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089810] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089817] Node 0 DMA32: 470*4kB 4*8kB 1*16kB 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3848kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089824] 1445 total pagecache pages
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089825] 0 pages in swap cache
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089826] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089827] Free swap = 0kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.089828] Total swap = 0kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095439] 985072 pages RAM
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095441] 24998 pages reserved
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095442] 10449 pages shared
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095443] 947553 pages non-shared
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095445] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095454] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095458] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095461] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095464] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095467] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095471] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095474] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095477] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095481] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095484] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095487] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095491] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095494] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095497] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095500] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095503] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095506] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095509] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095513] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095516] [ 3502] 1012 3502 71927 7760 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095519] [ 3504] 1012 3504 68832 6226 0 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095522] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095525] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095529] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095532] [ 6563] 0 6563 89584 2521 1 0 0 php5-fpm
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095535] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095538] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095541] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095544] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095547] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095550] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095553] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095556] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095560] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095563] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095566] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095569] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095572] [ 5438] 1002 5438 143346 79915 0 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095576] [25796] 1012 25796 585992 523101 0 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095578] [25811] 1012 25811 75323 12799 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095581] [25819] 1012 25819 273008 210123 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095584] [25861] 1012 25861 75013 12477 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095588] [25883] 1012 25883 119213 49359 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095591] [25961] 1012 25961 1099 25 0 0 0 sh
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095594] [25962] 1012 25962 18338 327 1 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095597] [25963] 1012 25963 1099 25 0 0 0 sh
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095600] [25964] 1012 25964 68832 6226 0 0 0 php
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095602] Out of memory: Kill process 25796 (php) score 546 or sacrifice child
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.095614] Killed process 25796 (php) total-vm:2343968kB, anon-rss:2092400kB, file-rss:4kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100116] php: page allocation failure: order:0, mode:0x280da
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100120] Pid: 25796, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100122] Call Trace:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100130] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100135] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100139] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100143] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100147] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100151] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100154] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100158] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100161] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100165] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100169] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100173] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100176] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100180] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100182] Mem-Info:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100184] Node 0 DMA per-cpu:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100186] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100188] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100189] Node 0 DMA32 per-cpu:
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100192] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100194] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100197] active_anon:930676 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100198] active_file:0 inactive_file:21 isolated_file:24
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100199] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100200] free:3166 slab_reclaimable:2536 slab_unreclaimable:3414
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100201] mapped:1359 shmem:1408 pagetables:3854 bounce:0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100203] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100211] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100215] Node 0 DMA32 free:4328kB min:7848kB low:9808kB high:11772kB active_anon:3722704kB inactive_anon:5584kB active_file:0kB inactive_file:84kB unevictable:0kB isolated(anon):0kB isolated(file):96kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5632kB slab_reclaimable:10144kB slab_unreclaimable:13656kB kernel_stack:872kB pagetables:15416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:18 all_unreclaimable? no
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100224] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100227] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100237] Node 0 DMA32: 526*4kB 34*8kB 2*16kB 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 4328kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100247] 1445 total pagecache pages
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100248] 0 pages in swap cache
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100250] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100251] Free swap = 0kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.100253] Total swap = 0kB
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.107432] 985072 pages RAM
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.107435] 24998 pages reserved
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.107436] 10449 pages shared
Jul 16 10:58:04 ip-10-61-174-46 kernel: [1499180.107437] 947553 pages non-shared
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249332] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249335] php5-fpm cpuset=/ mems_allowed=0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249338] Pid: 6563, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249340] Call Trace:
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249347] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249350] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249352] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249355] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249358] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249361] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249363] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249366] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249370] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249372] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249375] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249378] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249380] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249384] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249387] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249391] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249395] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249398] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249400] Mem-Info:
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249401] Node 0 DMA per-cpu:
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249403] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249404] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249405] Node 0 DMA32 per-cpu:
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249407] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249408] CPU 1: hi: 186, btch: 31 usd: 61
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249411] active_anon:928933 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249412] active_file:25 inactive_file:6 isolated_file:17
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249413] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249413] free:4006 slab_reclaimable:2549 slab_unreclaimable:3553
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249414] mapped:1371 shmem:1408 pagetables:4391 bounce:0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249416] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249422] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249425] Node 0 DMA32 free:7688kB min:7848kB low:9808kB high:11772kB active_anon:3715732kB inactive_anon:5584kB active_file:100kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):68kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5484kB shmem:5632kB slab_reclaimable:10196kB slab_unreclaimable:14212kB kernel_stack:960kB pagetables:17564kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:260 all_unreclaimable? yes
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249432] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249434] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249441] Node 0 DMA32: 1138*4kB 104*8kB 37*16kB 6*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 7704kB
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249448] 1449 total pagecache pages
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249449] 0 pages in swap cache
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249450] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249451] Free swap = 0kB
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.249452] Total swap = 0kB
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255195] 985072 pages RAM
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255197] 24998 pages reserved
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255198] 4227 pages shared
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255200] 952829 pages non-shared
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255201] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255215] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255219] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255222] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255225] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255229] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255232] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255235] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255239] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255243] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255246] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255249] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255252] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255256] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255259] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255262] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255265] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255268] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255271] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255274] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255278] [ 3502] 1012 3502 71927 7761 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255281] [ 3504] 1012 3504 68832 6228 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255284] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255287] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255290] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255293] [ 6563] 0 6563 89584 2505 1 0 0 php5-fpm
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255296] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255299] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255302] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255305] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255308] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255312] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255315] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255318] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255321] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255324] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255327] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255330] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255333] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255337] [25811] 1012 25811 127930 59008 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255340] [25819] 1012 25819 428921 366012 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255343] [25883] 1012 25883 352502 289624 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255346] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255349] [25973] 108 25973 6795 71 0 0 0 trivial-rewrite
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255352] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255355] [25987] 1012 25987 107920 44850 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255358] [26067] 1012 26067 75606 13033 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255362] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255365] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255368] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255371] [26148] 1012 26148 1099 24 0 0 0 sh
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255374] [26149] 1012 26149 68359 5795 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255377] [26159] 1012 26159 72097 9530 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255380] [26164] 1012 26164 70981 8426 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255383] [26173] 1012 26173 1099 24 0 0 0 sh
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255386] [26174] 1012 26174 67156 4620 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255389] [26186] 1012 26186 1099 24 0 0 0 sh
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255392] [26187] 1012 26187 64721 2626 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255395] [26188] 1012 26188 1099 25 1 0 0 sh
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255398] [26191] 1012 26191 64475 2063 1 0 0 php
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255401] Out of memory: Kill process 25819 (php) score 382 or sacrifice child
Jul 16 10:58:17 ip-10-61-174-46 kernel: [1499192.255412] Killed process 25819 (php) total-vm:1715684kB, anon-rss:1464048kB, file-rss:0kB
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228147] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228150] php cpuset=/ mems_allowed=0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228153] Pid: 5438, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228155] Call Trace:
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228161] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228164] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228166] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228169] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228174] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228176] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228185] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228187] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228191] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228193] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228196] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228199] [<ffffffff8164502b>] ? mm_fault_error+0x1/0xf3
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228202] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228204] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228206] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228209] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228211] Mem-Info:
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228212] Node 0 DMA per-cpu:
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228214] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228215] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228217] Node 0 DMA32 per-cpu:
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228218] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228219] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228222] active_anon:930659 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228223] active_file:18 inactive_file:18 isolated_file:0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228223] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228224] free:4047 slab_reclaimable:2519 slab_unreclaimable:3215
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228225] mapped:1362 shmem:1408 pagetables:4444 bounce:0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228226] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228233] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228236] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3722636kB inactive_anon:5584kB active_file:72kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5632kB slab_reclaimable:10076kB slab_unreclaimable:12860kB kernel_stack:992kB pagetables:17776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10817 all_unreclaimable? yes
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228242] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228245] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228252] Node 0 DMA32: 1120*4kB 179*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8008kB
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228258] 1447 total pagecache pages
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228259] 0 pages in swap cache
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228260] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228261] Free swap = 0kB
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.228262] Total swap = 0kB
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233867] 985072 pages RAM
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233869] 24998 pages reserved
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233870] 4246 pages shared
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233871] 952821 pages non-shared
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233873] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233882] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233886] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233889] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233892] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233895] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233899] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233902] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233905] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233909] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233912] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233915] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233919] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233922] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233925] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233928] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233931] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233934] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233938] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233941] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233944] [ 3502] 1012 3502 71927 7763 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233947] [ 3504] 1012 3504 68832 6233 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233950] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233953] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233956] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233959] [ 6563] 0 6563 89584 2541 1 0 0 php5-fpm
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233963] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233966] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233969] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233972] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233975] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233978] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233981] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233984] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233987] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233990] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233993] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233996] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.233999] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234003] [25811] 1012 25811 297925 235053 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234006] [25883] 1012 25883 494719 431891 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234009] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234012] [25973] 108 25973 6795 71 0 0 0 trivial-rewrite
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234015] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234018] [26067] 1012 26067 161762 98682 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234021] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234024] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234027] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234030] [26186] 1012 26186 1099 24 0 0 0 sh
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234033] [26187] 1012 26187 68636 6058 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234036] [26202] 1012 26202 72985 10430 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234039] [26264] 1012 26264 1099 25 0 0 0 sh
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234042] [26266] 1012 26266 68424 5869 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234045] [26324] 1012 26324 1099 25 1 0 0 sh
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234048] [26325] 1012 26325 4676 58 0 0 0 git
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234051] [26327] 1012 26327 68476 5481 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234054] [26340] 1012 26340 68170 5605 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234057] [26344] 1012 26344 68169 5594 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234060] [26347] 1012 26347 1099 24 1 0 0 sh
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234062] [26349] 1012 26349 63812 1624 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234066] [26353] 1012 26353 43650 872 0 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234069] [26354] 1012 26354 1099 25 1 0 0 sh
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234072] [26355] 1012 26355 11724 108 1 0 0 php
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234075] [26356] 1012 26356 1099 25 1 0 0 sh
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234078] Out of memory: Kill process 25883 (php) score 450 or sacrifice child
Jul 16 10:58:25 ip-10-61-174-46 kernel: [1499201.234088] Killed process 25883 (php) total-vm:1978876kB, anon-rss:1727564kB, file-rss:0kB
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252144] php cpuset=/ mems_allowed=0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252147] Pid: 26067, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252150] Call Trace:
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252161] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252164] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252167] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252171] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252176] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252183] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252188] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252191] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252195] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252198] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252201] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252205] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252207] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252209] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252212] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252215] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252218] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252220] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252223] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252226] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252227] Mem-Info:
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252229] Node 0 DMA per-cpu:
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252230] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252232] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252233] Node 0 DMA32 per-cpu:
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252236] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252238] active_anon:931250 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252239] active_file:16 inactive_file:29 isolated_file:0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252240] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252240] free:2850 slab_reclaimable:2528 slab_unreclaimable:3403
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252241] mapped:1363 shmem:1408 pagetables:4007 bounce:0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252249] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252252] Node 0 DMA32 free:3064kB min:7848kB low:9808kB high:11772kB active_anon:3725000kB inactive_anon:5584kB active_file:64kB inactive_file:116kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5632kB slab_reclaimable:10112kB slab_unreclaimable:13612kB kernel_stack:904kB pagetables:16028kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4605 all_unreclaimable? yes
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252259] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252261] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252268] Node 0 DMA32: 427*4kB 15*8kB 0*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 3108kB
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252275] 1428 total pagecache pages
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252276] 0 pages in swap cache
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252277] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252278] Free swap = 0kB
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.252279] Total swap = 0kB
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257974] 985072 pages RAM
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257977] 24998 pages reserved
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257978] 4233 pages shared
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257980] 954070 pages non-shared
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257981] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257990] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257993] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257996] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.257999] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258003] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258006] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258009] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258013] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258017] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258020] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258023] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258026] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258029] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258032] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258036] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258039] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258042] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258045] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258048] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258051] [ 3502] 1012 3502 71927 7766 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258055] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258058] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258061] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258064] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258067] [ 6563] 0 6563 89584 2538 0 0 0 php5-fpm
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258070] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258073] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258076] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258079] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258082] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258085] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258088] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258091] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258095] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258098] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258101] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258104] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258107] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258110] [25811] 1012 25811 449742 386881 1 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258113] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258116] [25973] 108 25973 6795 71 0 0 0 trivial-rewrite
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258120] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258122] [26067] 1012 26067 402049 339162 1 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258126] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258129] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258132] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258135] [26430] 1012 26430 127878 58914 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258137] [26473] 1012 26473 72033 9429 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258140] [26475] 1012 26475 71713 9134 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258143] [26563] 1012 26563 1099 24 0 0 0 sh
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258146] [26564] 1012 26564 66148 4102 0 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258149] [26568] 1012 26568 31064 393 1 0 0 php
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258152] Out of memory: Kill process 25811 (php) score 403 or sacrifice child
Jul 16 10:58:35 ip-10-61-174-46 kernel: [1499211.258162] Killed process 25811 (php) total-vm:1798968kB, anon-rss:1547524kB, file-rss:0kB
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816337] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816342] php cpuset=/ mems_allowed=0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816346] Pid: 26475, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816348] Call Trace:
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816356] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816359] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816362] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816366] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816371] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816375] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816380] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816382] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816386] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816389] [<ffffffff8113ca82>] ? do_wp_page+0x382/0x740
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816392] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816396] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816399] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816403] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816406] [<ffffffff8165d571>] ? down_read+0x1/0x20
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816409] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816413] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816417] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816420] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816424] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816427] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816431] Mem-Info:
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816432] Node 0 DMA per-cpu:
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816435] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816436] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816438] Node 0 DMA32 per-cpu:
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816440] CPU 0: hi: 186, btch: 31 usd: 32
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816446] CPU 1: hi: 186, btch: 31 usd: 33
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816449] active_anon:931591 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816450] active_file:26 inactive_file:8 isolated_file:0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816451] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816452] free:4035 slab_reclaimable:2464 slab_unreclaimable:3119
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816453] mapped:1361 shmem:1408 pagetables:3923 bounce:0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816455] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816463] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816467] Node 0 DMA32 free:7804kB min:7848kB low:9808kB high:11772kB active_anon:3726364kB inactive_anon:5584kB active_file:104kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5632kB slab_reclaimable:9856kB slab_unreclaimable:12476kB kernel_stack:888kB pagetables:15692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:763 all_unreclaimable? yes
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816476] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816479] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816489] Node 0 DMA32: 1417*4kB 1*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7772kB
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816498] 1442 total pagecache pages
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816500] 0 pages in swap cache
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816502] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816503] Free swap = 0kB
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.816504] Total swap = 0kB
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822340] 985072 pages RAM
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822342] 24998 pages reserved
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822343] 4235 pages shared
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822345] 952841 pages non-shared
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822347] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822355] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822358] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822361] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822365] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822368] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822371] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822374] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822378] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822382] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822385] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822388] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822391] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822394] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822397] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822400] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822403] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822406] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822409] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822413] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822416] [ 3502] 1012 3502 71927 7768 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822419] [ 3504] 1012 3504 68832 6235 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822422] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822425] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822428] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822431] [ 6563] 0 6563 89584 2512 0 0 0 php5-fpm
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822434] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822437] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822440] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822443] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822446] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822449] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822452] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822455] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822458] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822461] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822464] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822467] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822471] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822474] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822477] [25973] 108 25973 6795 71 0 0 0 trivial-rewrite
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822480] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822483] [26067] 1012 26067 579980 517125 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822486] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822489] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822492] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822495] [26430] 1012 26430 322424 259567 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822498] [26475] 1012 26475 72864 10281 1 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822501] [26628] 1012 26628 72069 9479 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822504] [26698] 1012 26698 70724 8191 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822507] [26718] 1012 26718 65830 3745 0 0 0 php
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822509] Out of memory: Kill process 26067 (php) score 539 or sacrifice child
Jul 16 10:58:45 ip-10-61-174-46 kernel: [1499220.822520] Killed process 26067 (php) total-vm:2319920kB, anon-rss:2068500kB, file-rss:0kB
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115976] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115980] ntpd cpuset=/ mems_allowed=0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115983] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115984] Call Trace:
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115992] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115994] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115996] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.115999] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116002] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116006] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116008] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116011] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116014] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116017] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116019] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116022] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116025] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116029] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116032] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116033] Mem-Info:
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116035] Node 0 DMA per-cpu:
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116037] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116038] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116039] Node 0 DMA32 per-cpu:
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116041] CPU 0: hi: 186, btch: 31 usd: 64
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116042] CPU 1: hi: 186, btch: 31 usd: 58
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116045] active_anon:931164 inactive_anon:1396 isolated_anon:0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116045] active_file:6 inactive_file:16 isolated_file:23
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116046] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116047] free:4021 slab_reclaimable:2456 slab_unreclaimable:3121
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116047] mapped:1392 shmem:1408 pagetables:4151 bounce:0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116049] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116070] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116075] Node 0 DMA32 free:7748kB min:7848kB low:9808kB high:11772kB active_anon:3724656kB inactive_anon:5584kB active_file:24kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):92kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5568kB shmem:5632kB slab_reclaimable:9824kB slab_unreclaimable:12484kB kernel_stack:920kB pagetables:16604kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:149 all_unreclaimable? yes
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116083] lowmem_reserve[]: 0 0 0 0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116087] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116097] Node 0 DMA32: 997*4kB 208*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7748kB
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116107] 1471 total pagecache pages
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116109] 0 pages in swap cache
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116111] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116112] Free swap = 0kB
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.116113] Total swap = 0kB
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121751] 985072 pages RAM
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121754] 24998 pages reserved
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121755] 10252 pages shared
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121757] 946774 pages non-shared
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121758] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121767] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121771] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121773] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121776] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121779] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121782] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121785] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121789] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121792] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121794] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121796] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121799] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121801] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121803] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121805] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121807] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121809] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121812] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121814] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121816] [ 3502] 1012 3502 71927 7774 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121818] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121821] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121823] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121825] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121827] [ 6563] 0 6563 89584 2529 0 0 0 php5-fpm
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121829] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121832] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121834] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121836] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121838] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121840] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121842] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121844] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121847] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121849] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121851] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121853] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121855] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121858] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121860] [25973] 108 25973 6795 75 1 0 0 trivial-rewrite
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121862] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121864] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121866] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121868] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121871] [26430] 1012 26430 701648 638825 0 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121873] [26882] 1012 26882 73725 11271 0 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121875] [26889] 1012 26889 72354 9805 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121877] [26910] 1012 26910 73533 10968 0 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121879] [26911] 1012 26911 204072 124923 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121881] [26968] 1012 26968 1099 25 1 0 0 sh
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121883] [26969] 1012 26969 68642 6074 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121885] [26970] 1012 26970 1099 25 1 0 0 sh
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121887] [26971] 1012 26971 68578 6023 1 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121889] [27051] 1012 27051 68578 6023 0 0 0 php
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121891] Out of memory: Kill process 26430 (php) score 666 or sacrifice child
Jul 16 10:58:59 ip-10-61-174-46 kernel: [1499235.121901] Killed process 26430 (php) total-vm:2806592kB, anon-rss:2555300kB, file-rss:0kB
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816165] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816170] ntpd cpuset=/ mems_allowed=0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816174] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816176] Call Trace:
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816183] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816187] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816190] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816193] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816197] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816202] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816205] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816209] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816212] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816215] [<ffffffff8113e861>] ? handle_pte_fault+0x1/0x200
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816219] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816224] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816229] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816233] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816235] Mem-Info:
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816237] Node 0 DMA per-cpu:
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816239] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816241] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816243] Node 0 DMA32 per-cpu:
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816245] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816247] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816250] active_anon:930817 inactive_anon:1396 isolated_anon:0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816251] active_file:8 inactive_file:0 isolated_file:0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816252] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816253] free:2849 slab_reclaimable:2470 slab_unreclaimable:3453
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816254] mapped:1356 shmem:1408 pagetables:4254 bounce:0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816256] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816265] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816269] Node 0 DMA32 free:3060kB min:7848kB low:9808kB high:11772kB active_anon:3723268kB inactive_anon:5584kB active_file:32kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5632kB slab_reclaimable:9880kB slab_unreclaimable:13812kB kernel_stack:952kB pagetables:17016kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:864 all_unreclaimable? yes
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816277] lowmem_reserve[]: 0 0 0 0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816281] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816290] Node 0 DMA32: 282*4kB 72*8kB 1*16kB 0*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 3064kB
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816299] 1429 total pagecache pages
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816301] 0 pages in swap cache
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816302] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816304] Free swap = 0kB
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.816305] Total swap = 0kB
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822026] 985072 pages RAM
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822029] 24998 pages reserved
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822030] 4231 pages shared
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822031] 954078 pages non-shared
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822033] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822041] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822044] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822047] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822051] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822054] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822057] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822060] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822064] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822067] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822070] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822073] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822077] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822080] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822083] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822086] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822089] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822092] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822095] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822098] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822101] [ 3502] 1012 3502 71927 7783 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822104] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822107] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822110] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822113] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822116] [ 6563] 0 6563 89584 2513 0 0 0 php5-fpm
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822119] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822123] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822126] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822129] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822132] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822135] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822138] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822141] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822144] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822147] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822150] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822153] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822156] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822160] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822163] [25973] 108 25973 6795 75 0 0 0 trivial-rewrite
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822166] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822169] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822172] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822175] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822178] [26911] 1012 26911 731917 669067 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822181] [27248] 1012 27248 71693 9160 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822184] [27286] 1012 27286 155982 93137 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822187] [27367] 1012 27367 70981 8428 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822189] [27400] 1012 27400 1099 24 0 0 0 sh
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822192] [27401] 1012 27401 68424 5868 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822195] [27403] 1012 27403 70314 7727 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822198] [27411] 1012 27411 1099 26 1 0 0 sh
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822201] [27412] 1012 27412 4724 58 0 0 0 git
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822204] [27413] 1012 27413 68476 5522 1 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822207] [27417] 1012 27417 67117 4587 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822210] [27419] 1012 27419 65829 3756 0 0 0 php
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822213] [27420] 1012 27420 1099 24 1 0 0 sh
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822216] [27421] 1012 27421 10901 167 0 0 0 ssh
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822218] Out of memory: Kill process 26911 (php) score 698 or sacrifice child
Jul 16 10:59:18 ip-10-61-174-46 kernel: [1499253.822229] Killed process 26911 (php) total-vm:2927668kB, anon-rss:2676268kB, file-rss:0kB
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816139] php cpuset=/ mems_allowed=0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816142] Pid: 5438, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816144] Call Trace:
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816152] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816159] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816167] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816171] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816183] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816186] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816191] [<ffffffff81006351>] ? pte_mfn_to_pfn+0x81/0x110
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816197] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816203] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816206] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816210] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816213] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816217] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816221] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816224] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816226] Mem-Info:
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816228] Node 0 DMA per-cpu:
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816230] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816231] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816232] Node 0 DMA32 per-cpu:
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816235] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816237] active_anon:930609 inactive_anon:1396 isolated_anon:0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816238] active_file:0 inactive_file:24 isolated_file:0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816239] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816239] free:3225 slab_reclaimable:2391 slab_unreclaimable:3482
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816240] mapped:1355 shmem:1408 pagetables:3934 bounce:0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816248] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816251] Node 0 DMA32 free:4564kB min:7848kB low:9808kB high:11772kB active_anon:3722436kB inactive_anon:5584kB active_file:0kB inactive_file:96kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9564kB slab_unreclaimable:13928kB kernel_stack:896kB pagetables:15736kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1784 all_unreclaimable? yes
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816258] lowmem_reserve[]: 0 0 0 0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816260] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816267] Node 0 DMA32: 691*4kB 111*8kB 1*16kB 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 4564kB
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816273] 1426 total pagecache pages
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816275] 0 pages in swap cache
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816276] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816277] Free swap = 0kB
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.816278] Total swap = 0kB
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822400] 985072 pages RAM
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822402] 24998 pages reserved
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822403] 10055 pages shared
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822405] 947885 pages non-shared
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822407] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822415] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822418] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822422] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822425] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822428] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822431] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822435] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822438] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822442] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822445] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822448] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822451] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822454] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822457] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822461] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822464] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822467] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822469] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822473] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822476] [ 3502] 1012 3502 71927 7793 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822479] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822482] [ 3506] 1012 3506 69239 6590 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822485] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822488] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822491] [ 6563] 0 6563 89584 2497 0 0 0 php5-fpm
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822494] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822497] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822500] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822503] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822506] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822509] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822512] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822515] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822518] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822521] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822524] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822527] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822530] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822534] [25972] 108 25972 6818 73 0 0 0 cleanup
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822537] [25973] 108 25973 6795 75 0 0 0 trivial-rewrite
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822540] [25976] 108 25976 11107 158 0 0 0 smtp
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822543] [26081] 108 26081 11107 158 1 0 0 smtp
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822546] [26089] 108 26089 11107 158 1 0 0 smtp
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822549] [26091] 108 26091 11107 158 0 0 0 smtp
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822552] [27286] 1012 27286 818254 755376 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822555] [27767] 1012 27767 72135 9496 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822558] [27813] 1012 27813 72069 9487 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822561] [27877] 1012 27877 91866 27040 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822564] [27887] 1012 27887 1099 24 0 0 0 sh
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822567] [27888] 1012 27888 68424 5870 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822569] [27900] 1012 27900 68424 5870 0 0 0 php
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822572] Out of memory: Kill process 27286 (php) score 788 or sacrifice child
Jul 16 10:59:40 ip-10-61-174-46 kernel: [1499275.822584] Killed process 27286 (php) total-vm:3273016kB, anon-rss:3021504kB, file-rss:0kB
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872192] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872197] php cpuset=/ mems_allowed=0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872200] Pid: 27877, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872202] Call Trace:
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872210] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872214] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872216] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872220] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872224] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872229] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872232] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872236] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872240] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872243] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872247] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872250] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872253] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872256] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872259] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872266] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872270] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872274] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872278] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872281] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872284] Mem-Info:
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872285] Node 0 DMA per-cpu:
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872287] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872289] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872290] Node 0 DMA32 per-cpu:
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872292] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872294] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872298] active_anon:931371 inactive_anon:1396 isolated_anon:0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872299] active_file:0 inactive_file:0 isolated_file:0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872300] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872300] free:2852 slab_reclaimable:2437 slab_unreclaimable:3440
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872301] mapped:1355 shmem:1408 pagetables:3711 bounce:0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872304] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872312] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872318] Node 0 DMA32 free:3072kB min:7848kB low:9808kB high:11772kB active_anon:3725484kB inactive_anon:5584kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9748kB slab_unreclaimable:13760kB kernel_stack:840kB pagetables:14844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872326] lowmem_reserve[]: 0 0 0 0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872329] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872339] Node 0 DMA32: 345*4kB 17*8kB 1*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3164kB
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872349] 1434 total pagecache pages
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872350] 0 pages in swap cache
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872352] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872353] Free swap = 0kB
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.872356] Total swap = 0kB
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879631] 985072 pages RAM
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879633] 24998 pages reserved
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879634] 4245 pages shared
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879636] 954053 pages non-shared
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879637] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879646] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879650] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879653] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879656] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879659] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879663] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879666] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879669] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879673] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879676] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879679] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879683] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879686] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879689] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879692] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879695] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879698] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879702] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879705] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879708] [ 3502] 1012 3502 71927 7806 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879712] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879715] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879718] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879721] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879724] [ 6563] 0 6563 89584 2528 0 0 0 php5-fpm
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879727] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879730] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879733] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879736] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879739] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879742] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879745] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879748] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879751] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879754] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879757] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879760] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879763] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879767] [27877] 1012 27877 806288 743398 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879770] [28440] 1012 28440 72145 9572 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879773] [28481] 1012 28481 112461 43909 0 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879776] [28513] 1012 28513 72081 9532 1 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879779] [28574] 1012 28574 1099 24 0 0 0 sh
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879782] [28575] 1012 28575 64458 2434 0 0 0 php
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879784] Out of memory: Kill process 27877 (php) score 775 or sacrifice child
Jul 16 11:00:05 ip-10-61-174-46 kernel: [1499300.879796] Killed process 27877 (php) total-vm:3225152kB, anon-rss:2973592kB, file-rss:0kB
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819186] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819191] php cpuset=/ mems_allowed=0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819194] Pid: 29028, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819196] Call Trace:
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819204] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819207] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819210] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819214] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819218] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819222] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819225] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819229] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819233] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819236] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819240] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819243] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819247] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819252] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819256] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819259] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819263] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819266] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819270] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819274] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819276] Mem-Info:
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819278] Node 0 DMA per-cpu:
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819280] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819282] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819284] Node 0 DMA32 per-cpu:
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819286] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819288] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819291] active_anon:930624 inactive_anon:1396 isolated_anon:0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819292] active_file:3 inactive_file:7 isolated_file:0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819293] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819294] free:3834 slab_reclaimable:2758 slab_unreclaimable:3237
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819295] mapped:1383 shmem:1408 pagetables:3982 bounce:0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819297] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819306] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819310] Node 0 DMA32 free:7000kB min:7848kB low:9808kB high:11772kB active_anon:3722496kB inactive_anon:5584kB active_file:12kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5532kB shmem:5632kB slab_reclaimable:11032kB slab_unreclaimable:12948kB kernel_stack:872kB pagetables:15928kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:63 all_unreclaimable? yes
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819318] lowmem_reserve[]: 0 0 0 0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819321] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819331] Node 0 DMA32: 460*4kB 254*8kB 59*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 6896kB
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819340] 1449 total pagecache pages
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819341] 0 pages in swap cache
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819343] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819345] Free swap = 0kB
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.819346] Total swap = 0kB
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826764] 985072 pages RAM
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826767] 24998 pages reserved
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826768] 4339 pages shared
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826769] 953505 pages non-shared
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826772] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826780] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826784] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826787] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826790] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826793] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826797] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826800] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826803] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826807] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826810] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826813] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826836] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826840] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826842] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826845] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826848] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826852] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826855] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826859] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826863] [ 3502] 1012 3502 71991 7824 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826866] [ 3504] 1012 3504 68832 6242 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826869] [ 3506] 1012 3506 69239 6590 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826872] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826875] [ 3533] 1012 3533 68271 5641 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826878] [ 6563] 0 6563 89584 2512 1 0 0 php5-fpm
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826881] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826884] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826887] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826890] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826893] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826896] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826899] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826902] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826904] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826908] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826911] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826914] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826917] [ 5438] 1002 5438 143346 79917 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826943] [28481] 1012 28481 797201 734322 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826946] [28958] 1012 28958 72005 9453 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826949] [28977] 1012 28977 110281 42030 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826953] [29022] 1012 29022 1099 25 1 0 0 sh
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826956] [29023] 1012 29023 66830 4297 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826959] [29028] 1012 29028 68169 5601 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826961] [29030] 1012 29030 68169 5588 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826965] [29032] 1012 29032 66341 4283 1 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826967] [29035] 1012 29035 1099 25 0 0 0 sh
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826986] [29038] 1012 29038 64458 2435 0 0 0 php
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.826988] Out of memory: Kill process 28481 (php) score 766 or sacrifice child
Jul 16 11:00:26 ip-10-61-174-46 kernel: [1499321.827001] Killed process 28481 (php) total-vm:3188804kB, anon-rss:2937260kB, file-rss:28kB
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579490] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579494] php5-fpm cpuset=/ mems_allowed=0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579497] Pid: 6563, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579499] Call Trace:
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579506] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579508] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579510] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579513] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579516] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579520] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579522] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579525] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579528] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579531] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579534] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579536] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579539] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579542] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579545] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579549] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579553] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579556] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579558] Mem-Info:
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579559] Node 0 DMA per-cpu:
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579561] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579563] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579564] Node 0 DMA32 per-cpu:
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579565] CPU 0: hi: 186, btch: 31 usd: 87
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579567] CPU 1: hi: 186, btch: 31 usd: 49
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579569] active_anon:931547 inactive_anon:1396 isolated_anon:0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579570] active_file:42 inactive_file:8 isolated_file:23
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579571] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579571] free:4026 slab_reclaimable:2379 slab_unreclaimable:3107
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579572] mapped:1359 shmem:1408 pagetables:3920 bounce:0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579574] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579580] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579583] Node 0 DMA32 free:7768kB min:7848kB low:9808kB high:11772kB active_anon:3726188kB inactive_anon:5584kB active_file:168kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):92kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5632kB slab_reclaimable:9516kB slab_unreclaimable:12428kB kernel_stack:872kB pagetables:15680kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:480 all_unreclaimable? yes
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579590] lowmem_reserve[]: 0 0 0 0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579592] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579599] Node 0 DMA32: 1212*4kB 113*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579606] 1466 total pagecache pages
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579607] 0 pages in swap cache
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579608] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579610] Free swap = 0kB
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.579610] Total swap = 0kB
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585140] 985072 pages RAM
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585142] 24998 pages reserved
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585143] 4231 pages shared
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585144] 952767 pages non-shared
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585146] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585155] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585159] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585162] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585165] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585168] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585172] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585175] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585178] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585182] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585185] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585189] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585192] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585195] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585198] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585202] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585205] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585208] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585211] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585214] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585218] [ 3502] 1012 3502 71991 7825 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585221] [ 3504] 1012 3504 68832 6236 0 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585224] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585227] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585230] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585233] [ 6563] 0 6563 89584 2496 1 0 0 php5-fpm
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585236] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585239] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585242] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585245] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585248] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585251] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585254] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585257] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585261] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585263] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585266] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585270] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585273] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585277] [28977] 1012 28977 670533 607653 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585279] [29222] 1012 29222 229013 165329 0 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585282] [29332] 1012 29332 58623 8232 0 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585286] [29348] 1012 29348 71748 9223 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585289] [29352] 1012 29352 70713 8182 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585292] [29369] 1012 29369 72033 9469 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585294] [29430] 1012 29430 1099 25 0 0 0 sh
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585298] [29431] 1012 29431 39367 452 0 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585301] [29432] 1012 29432 1099 24 0 0 0 sh
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585304] [29435] 1012 29435 39367 454 1 0 0 php
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585307] Out of memory: Kill process 28977 (php) score 634 or sacrifice child
Jul 16 11:00:45 ip-10-61-174-46 kernel: [1499340.585318] Killed process 28977 (php) total-vm:2682132kB, anon-rss:2430612kB, file-rss:0kB
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888333] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888337] php cpuset=/ mems_allowed=0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888340] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888342] Call Trace:
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888349] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888351] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888353] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888356] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888360] [<ffffffff81312aa6>] ? radix_tree_next_hole+0x36/0x50
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888363] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888367] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888369] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888372] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888374] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888378] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888380] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888382] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888386] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888389] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888393] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888396] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888399] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888400] Mem-Info:
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888401] Node 0 DMA per-cpu:
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888403] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888405] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888406] Node 0 DMA32 per-cpu:
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888407] CPU 0: hi: 186, btch: 31 usd: 132
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888409] CPU 1: hi: 186, btch: 31 usd: 27
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888411] active_anon:931332 inactive_anon:1396 isolated_anon:0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888412] active_file:0 inactive_file:0 isolated_file:0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888413] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888413] free:4039 slab_reclaimable:2556 slab_unreclaimable:3087
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888414] mapped:1356 shmem:1408 pagetables:3903 bounce:0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888416] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888422] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888425] Node 0 DMA32 free:7820kB min:7848kB low:9808kB high:11772kB active_anon:3725328kB inactive_anon:5584kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5632kB slab_reclaimable:10224kB slab_unreclaimable:12348kB kernel_stack:872kB pagetables:15612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:309 all_unreclaimable? yes
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888432] lowmem_reserve[]: 0 0 0 0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888434] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888441] Node 0 DMA32: 1073*4kB 179*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7820kB
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888448] 1433 total pagecache pages
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888449] 0 pages in swap cache
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888450] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888451] Free swap = 0kB
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.888452] Total swap = 0kB
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893918] 985072 pages RAM
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893921] 24998 pages reserved
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893922] 4234 pages shared
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893923] 952736 pages non-shared
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893925] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893934] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893937] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893941] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893944] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893947] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893950] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893953] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893957] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893961] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893964] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893967] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893970] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893974] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893977] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893980] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893983] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893986] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893989] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893992] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893996] [ 3502] 1012 3502 71991 7830 1 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.893999] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894002] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894005] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894008] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894011] [ 6563] 0 6563 89584 2534 1 0 0 php5-fpm
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894014] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894017] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894020] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894023] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894027] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894030] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894033] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894036] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894039] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894042] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894045] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894048] [29474] 108 29474 6792 76 0 0 0 pickup
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894051] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894055] [29222] 1012 29222 645191 582317 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894057] [29369] 1012 29369 72225 9649 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894060] [29608] 1012 29608 255224 192393 1 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894064] [29709] 1012 29709 71045 8453 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894067] [29737] 1012 29737 71961 9399 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894070] [29755] 1012 29755 1099 25 0 0 0 sh
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894072] [29756] 1012 29756 68424 5869 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894075] [29777] 1012 29777 1099 26 0 0 0 sh
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894078] [29778] 1012 29778 14084 62 0 0 0 git
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894082] [29780] 1012 29780 43844 651 0 0 0 php
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894084] Out of memory: Kill process 29222 (php) score 607 or sacrifice child
Jul 16 11:01:00 ip-10-61-174-46 kernel: [1499355.894095] Killed process 29222 (php) total-vm:2580764kB, anon-rss:2329268kB, file-rss:0kB
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292554] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292559] php cpuset=/ mems_allowed=0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292562] Pid: 30061, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292564] Call Trace:
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292571] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292574] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292576] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292579] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292582] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292585] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292587] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292591] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292595] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292597] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292600] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292603] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292605] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292608] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292610] [<ffffffff811425b1>] ? find_vma+0x1/0x70
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292613] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292615] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292617] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292619] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292622] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292624] Mem-Info:
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292625] Node 0 DMA per-cpu:
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292627] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292628] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292629] Node 0 DMA32 per-cpu:
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292631] CPU 0: hi: 186, btch: 31 usd: 21
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292632] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292635] active_anon:931179 inactive_anon:1396 isolated_anon:0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292635] active_file:5 inactive_file:11 isolated_file:0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292636] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292637] free:4055 slab_reclaimable:2507 slab_unreclaimable:3141
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292637] mapped:1361 shmem:1408 pagetables:4104 bounce:0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292639] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292645] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292648] Node 0 DMA32 free:7884kB min:7848kB low:9808kB high:11772kB active_anon:3724716kB inactive_anon:5584kB active_file:20kB inactive_file:44kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5632kB slab_reclaimable:10028kB slab_unreclaimable:12564kB kernel_stack:984kB pagetables:16416kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:118 all_unreclaimable? yes
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292663] lowmem_reserve[]: 0 0 0 0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292667] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292675] Node 0 DMA32: 559*4kB 442*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7852kB
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292685] 1489 total pagecache pages
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292686] 0 pages in swap cache
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292688] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292689] Free swap = 0kB
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.292691] Total swap = 0kB
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298183] 985072 pages RAM
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298186] 24998 pages reserved
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298187] 4250 pages shared
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298188] 952799 pages non-shared
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298190] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298199] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298202] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298205] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298208] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298212] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298215] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298218] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298222] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298226] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298229] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298232] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298235] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298238] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298241] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298245] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298247] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298251] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298254] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298257] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298260] [ 3502] 1012 3502 71991 7840 1 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298263] [ 3504] 1012 3504 68832 6236 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298266] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298269] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298272] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298276] [ 6563] 0 6563 89584 2499 0 0 0 php5-fpm
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298279] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298282] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298285] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298288] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298291] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298295] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298298] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298301] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298305] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298309] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298311] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298315] [ 5438] 1002 5438 143346 79925 1 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298319] [29608] 1012 29608 839187 776334 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298322] [29842] 108 29842 6818 72 0 0 0 cleanup
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298325] [29843] 108 29843 6795 71 1 0 0 trivial-rewrite
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298328] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298331] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298334] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298336] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298340] [29864] 108 29864 6792 69 0 0 0 pickup
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298342] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298346] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298349] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298351] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298354] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298357] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298360] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298363] [30061] 1012 30061 40276 7580 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298366] [30127] 1012 30127 72167 9582 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298370] [30155] 1012 30155 1099 24 0 0 0 sh
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298373] [30156] 1012 30156 68424 5871 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298376] [30168] 1012 30168 1099 26 0 0 0 sh
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298378] [30169] 1012 30169 14084 64 1 0 0 git
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298381] [30170] 1012 30170 68476 5522 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298384] [30176] 1012 30176 63874 1683 0 0 0 php
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298387] [30181] 1012 30181 1099 24 1 0 0 sh
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298390] [30182] 1012 30182 10901 175 0 0 0 ssh
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298393] Out of memory: Kill process 29608 (php) score 810 or sacrifice child
Jul 16 11:01:19 ip-10-61-174-46 kernel: [1499375.298405] Killed process 29608 (php) total-vm:3356748kB, anon-rss:3105336kB, file-rss:0kB
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024622] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024627] php5-fpm cpuset=/ mems_allowed=0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024630] Pid: 6563, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024632] Call Trace:
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024640] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024644] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024647] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024651] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024655] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024659] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024662] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024666] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024671] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024674] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024678] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024681] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024684] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024688] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024692] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024697] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024702] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024706] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024708] Mem-Info:
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024710] Node 0 DMA per-cpu:
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024712] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024714] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024715] Node 0 DMA32 per-cpu:
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024717] CPU 0: hi: 186, btch: 31 usd: 84
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024719] CPU 1: hi: 186, btch: 31 usd: 60
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024723] active_anon:930205 inactive_anon:1396 isolated_anon:0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024725] active_file:26 inactive_file:2 isolated_file:18
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024726] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024727] free:4042 slab_reclaimable:2596 slab_unreclaimable:3235
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024728] mapped:1390 shmem:1408 pagetables:4637 bounce:0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024731] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024740] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024744] Node 0 DMA32 free:7832kB min:7848kB low:9808kB high:11772kB active_anon:3720820kB inactive_anon:5584kB active_file:104kB inactive_file:8kB unevictable:0kB isolated(anon):0kB isolated(file):72kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5560kB shmem:5632kB slab_reclaimable:10384kB slab_unreclaimable:12940kB kernel_stack:1064kB pagetables:18548kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:260 all_unreclaimable? yes
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024752] lowmem_reserve[]: 0 0 0 0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024756] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024765] Node 0 DMA32: 802*4kB 174*8kB 66*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7832kB
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024774] 1427 total pagecache pages
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024775] 0 pages in swap cache
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024777] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024779] Free swap = 0kB
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.024780] Total swap = 0kB
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030291] 985072 pages RAM
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030293] 24998 pages reserved
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030295] 4264 pages shared
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030296] 952688 pages non-shared
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030298] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030306] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030310] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030313] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030316] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030319] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030323] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030326] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030330] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030334] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030337] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030340] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030343] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030346] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030350] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030353] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030356] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030359] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030362] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030366] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030369] [ 3502] 1012 3502 71991 7850 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030372] [ 3504] 1012 3504 68832 6236 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030375] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030378] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030381] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030384] [ 6563] 0 6563 89584 2533 1 0 0 php5-fpm
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030387] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030391] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030394] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030397] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030400] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030403] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030406] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030409] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030412] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030415] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030418] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030421] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030425] [29842] 108 29842 6818 72 0 0 0 cleanup
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030428] [29843] 108 29843 6795 71 0 0 0 trivial-rewrite
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030431] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030434] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030437] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030440] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030443] [29864] 108 29864 6792 69 0 0 0 pickup
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030446] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030449] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030453] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030455] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030458] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030461] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030464] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030468] [30204] 1012 30204 609857 547022 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030471] [30468] 1012 30468 266301 203456 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030474] [30574] 1012 30574 72723 10131 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030477] [30577] 1012 30577 72089 9481 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030480] [30615] 1012 30615 1099 25 1 0 0 sh
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030483] [30616] 1012 30616 68424 5870 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030486] [30617] 1012 30617 1099 25 0 0 0 sh
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030489] [30620] 1012 30620 68424 5868 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030492] [30621] 1012 30621 1099 25 0 0 0 sh
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030495] [30622] 1012 30622 68424 5869 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030498] [30632] 1012 30632 72017 9486 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030501] [30671] 1012 30671 1099 26 1 0 0 sh
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030504] [30672] 1012 30672 4738 67 0 0 0 git
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030507] [30676] 1012 30676 65638 3173 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030510] [30681] 1012 30681 1099 26 0 0 0 sh
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030513] [30684] 1012 30684 4649 57 1 0 0 git
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030516] [30686] 1012 30686 65182 2746 0 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030519] [30689] 1012 30689 1099 25 0 0 0 sh
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030522] [30690] 1012 30690 14084 63 0 0 0 git
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030525] [30695] 1012 30695 64857 2439 1 0 0 php
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030528] Out of memory: Kill process 30204 (php) score 570 or sacrifice child
Jul 16 11:01:41 ip-10-61-174-46 kernel: [1499397.030539] Killed process 30204 (php) total-vm:2439428kB, anon-rss:2188088kB, file-rss:0kB
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550146] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550150] php cpuset=/ mems_allowed=0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550153] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550155] Call Trace:
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550162] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550164] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550167] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550169] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550174] [<ffffffff81312aa6>] ? radix_tree_next_hole+0x36/0x50
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550177] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550185] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550191] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550193] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550196] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550203] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550206] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550209] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550212] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550214] Mem-Info:
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550215] Node 0 DMA per-cpu:
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550217] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550218] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550219] Node 0 DMA32 per-cpu:
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550221] CPU 0: hi: 186, btch: 31 usd: 51
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550222] CPU 1: hi: 186, btch: 31 usd: 71
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550225] active_anon:931380 inactive_anon:1395 isolated_anon:0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550226] active_file:17 inactive_file:12 isolated_file:20
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550226] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550227] free:4035 slab_reclaimable:2561 slab_unreclaimable:3106
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550228] mapped:1379 shmem:1408 pagetables:3903 bounce:0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550229] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550236] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550239] Node 0 DMA32 free:7804kB min:7848kB low:9808kB high:11772kB active_anon:3725520kB inactive_anon:5580kB active_file:68kB inactive_file:48kB unevictable:0kB isolated(anon):0kB isolated(file):80kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5632kB slab_reclaimable:10244kB slab_unreclaimable:12424kB kernel_stack:944kB pagetables:15612kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:296 all_unreclaimable? yes
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550246] lowmem_reserve[]: 0 0 0 0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550248] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550255] Node 0 DMA32: 1217*4kB 105*8kB 3*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7804kB
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550261] 1440 total pagecache pages
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550263] 0 pages in swap cache
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550264] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550265] Free swap = 0kB
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.550266] Total swap = 0kB
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555768] 985072 pages RAM
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555770] 24998 pages reserved
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555772] 4241 pages shared
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555773] 952761 pages non-shared
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555775] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555784] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555787] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555790] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555794] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555797] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555800] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555803] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555807] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555811] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555814] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555817] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555820] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555823] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555826] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555829] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555833] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555836] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555839] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555842] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555845] [ 3502] 1012 3502 71991 7858 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555848] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555851] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555854] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555857] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555861] [ 6563] 0 6563 89584 2499 1 0 0 php5-fpm
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555864] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555867] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555870] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555873] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555876] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555879] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555882] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555885] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555888] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555892] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555895] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555898] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555901] [29842] 108 29842 6818 72 0 0 0 cleanup
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555904] [29843] 108 29843 6795 71 0 0 0 trivial-rewrite
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555908] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555910] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555913] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555916] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555919] [29864] 108 29864 6792 69 1 0 0 pickup
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555922] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555926] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555929] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555932] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555934] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555937] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555941] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555944] [30468] 1012 30468 572807 509955 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555947] [30808] 1012 30808 353204 290298 0 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555950] [31039] 1012 31039 1099 24 0 0 0 sh
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555953] [31040] 1012 31040 67286 4718 0 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555956] [31046] 1012 31046 64279 1991 1 0 0 php
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555959] Out of memory: Kill process 30468 (php) score 532 or sacrifice child
Jul 16 11:01:57 ip-10-61-174-46 kernel: [1499412.555970] Killed process 30468 (php) total-vm:2291228kB, anon-rss:2039820kB, file-rss:0kB
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644321] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644325] php cpuset=/ mems_allowed=0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644328] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644329] Call Trace:
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644337] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644339] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644341] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644344] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644349] [<ffffffff81312aa6>] ? radix_tree_next_hole+0x36/0x50
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644352] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644355] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644357] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644360] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644363] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644366] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644368] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644371] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644374] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644378] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644382] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644384] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644387] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644389] Mem-Info:
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644390] Node 0 DMA per-cpu:
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644392] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644394] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644395] Node 0 DMA32 per-cpu:
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644396] CPU 0: hi: 186, btch: 31 usd: 85
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644398] CPU 1: hi: 186, btch: 31 usd: 25
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644400] active_anon:931040 inactive_anon:1396 isolated_anon:0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644401] active_file:18 inactive_file:18 isolated_file:24
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644402] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644402] free:4031 slab_reclaimable:2520 slab_unreclaimable:3178
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644403] mapped:1349 shmem:1408 pagetables:4155 bounce:0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644405] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644411] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644414] Node 0 DMA32 free:7788kB min:7848kB low:9808kB high:11772kB active_anon:3724160kB inactive_anon:5584kB active_file:72kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):96kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5396kB shmem:5632kB slab_reclaimable:10080kB slab_unreclaimable:12712kB kernel_stack:1000kB pagetables:16620kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:287 all_unreclaimable? yes
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644421] lowmem_reserve[]: 0 0 0 0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644423] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644430] Node 0 DMA32: 693*4kB 373*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7804kB
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644437] 1433 total pagecache pages
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644438] 0 pages in swap cache
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644439] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644440] Free swap = 0kB
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.644441] Total swap = 0kB
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650053] 985072 pages RAM
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650056] 24998 pages reserved
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650057] 4241 pages shared
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650058] 952766 pages non-shared
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650060] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650068] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650072] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650075] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650078] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650081] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650085] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650088] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650091] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650095] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650098] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650101] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650105] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650108] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650111] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650114] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650117] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650120] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650123] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650127] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650130] [ 3502] 1012 3502 71991 7862 0 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650133] [ 3504] 1012 3504 68832 6236 0 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650136] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650139] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650142] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650145] [ 6563] 0 6563 89584 2527 1 0 0 php5-fpm
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650149] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650152] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650155] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650158] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650161] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650164] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650167] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650170] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650173] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650176] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650179] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650183] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650186] [29842] 108 29842 6818 72 0 0 0 cleanup
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650189] [29843] 108 29843 6795 75 0 0 0 trivial-rewrite
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650192] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650195] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650198] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650201] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650204] [29864] 108 29864 6792 69 1 0 0 pickup
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650207] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650210] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650213] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650216] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650219] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650222] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650225] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650229] [30808] 1012 30808 615620 552713 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650232] [31046] 1012 31046 285422 222530 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650235] [31101] 1012 31101 72909 10374 0 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650238] [31131] 1012 31131 71252 9292 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650241] [31205] 1012 31205 1099 25 1 0 0 sh
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650244] [31206] 1012 31206 68424 5868 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650247] [31226] 1012 31226 1099 26 0 0 0 sh
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650250] [31227] 1012 31227 4777 63 0 0 0 git
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650253] [31228] 1012 31228 68476 5522 1 0 0 php
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650256] [31238] 1012 31238 1099 25 1 0 0 sh
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650259] [31239] 1012 31239 10901 164 1 0 0 ssh
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650262] [31247] 1012 31247 4647 43 0 0 0 git
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650264] Out of memory: Kill process 30808 (php) score 576 or sacrifice child
Jul 16 11:02:08 ip-10-61-174-46 kernel: [1499423.650275] Killed process 30808 (php) total-vm:2462480kB, anon-rss:2210852kB, file-rss:0kB
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868789] ssh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868793] ssh cpuset=/ mems_allowed=0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868796] Pid: 31503, comm: ssh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868798] Call Trace:
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868805] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868807] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868809] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868812] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868816] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868819] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868821] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868824] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868827] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868830] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868832] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868835] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868839] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868841] [<ffffffff81142610>] ? find_vma+0x60/0x70
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868844] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868848] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868851] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868853] Mem-Info:
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868854] Node 0 DMA per-cpu:
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868856] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868857] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868858] Node 0 DMA32 per-cpu:
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868860] CPU 0: hi: 186, btch: 31 usd: 92
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868861] CPU 1: hi: 186, btch: 31 usd: 31
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868864] active_anon:930955 inactive_anon:1396 isolated_anon:0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868865] active_file:21 inactive_file:27 isolated_file:23
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868865] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868866] free:3989 slab_reclaimable:2450 slab_unreclaimable:3141
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868867] mapped:1385 shmem:1408 pagetables:4305 bounce:0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868868] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868875] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868878] Node 0 DMA32 free:7620kB min:7848kB low:9808kB high:11772kB active_anon:3723820kB inactive_anon:5584kB active_file:84kB inactive_file:108kB unevictable:0kB isolated(anon):0kB isolated(file):92kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5540kB shmem:5632kB slab_reclaimable:9800kB slab_unreclaimable:12564kB kernel_stack:1040kB pagetables:17220kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:288 all_unreclaimable? yes
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868884] lowmem_reserve[]: 0 0 0 0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868887] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868893] Node 0 DMA32: 787*4kB 303*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7620kB
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868900] 1462 total pagecache pages
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868901] 0 pages in swap cache
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868903] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868904] Free swap = 0kB
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.868905] Total swap = 0kB
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874488] 985072 pages RAM
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874491] 24998 pages reserved
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874492] 4240 pages shared
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874493] 952795 pages non-shared
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874495] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874504] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874507] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874511] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874514] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874517] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874520] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874523] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874527] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874531] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874534] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874541] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874544] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874547] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874551] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874554] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874557] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874560] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874563] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874566] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874570] [ 3502] 1012 3502 71991 7866 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874575] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874578] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874583] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874586] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874589] [ 6563] 0 6563 89584 2531 0 0 0 php5-fpm
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874593] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874596] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874599] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874602] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874605] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874608] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874612] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874615] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874618] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874621] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874624] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874627] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874631] [29842] 108 29842 6818 72 0 0 0 cleanup
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874634] [29843] 108 29843 6795 75 1 0 0 trivial-rewrite
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874637] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874640] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874643] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874646] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874649] [29864] 108 29864 6792 69 1 0 0 pickup
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874652] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874655] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874658] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874661] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874664] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874667] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874670] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874673] [31046] 1012 31046 592833 530005 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874676] [31254] 1012 31254 305459 242603 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874679] [31401] 1012 31401 73041 10541 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874683] [31452] 1012 31452 1099 24 0 0 0 sh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874686] [31453] 1012 31453 68424 5869 1 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874689] [31454] 1012 31454 1099 24 1 0 0 sh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874692] [31456] 1012 31456 68424 5869 1 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874695] [31482] 1012 31482 1099 25 0 0 0 sh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874698] [31483] 1012 31483 4703 67 0 0 0 git
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874701] [31485] 1012 31485 1099 26 1 0 0 sh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874704] [31487] 1012 31487 5216 83 0 0 0 git
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874707] [31488] 1012 31488 68476 5523 1 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874710] [31489] 1012 31489 68476 5522 0 0 0 php
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874713] [31498] 1012 31498 1099 26 0 0 0 sh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874716] [31499] 1012 31499 10901 176 0 0 0 ssh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874719] [31502] 1012 31502 1099 25 1 0 0 sh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874721] [31503] 1012 31503 10901 175 1 0 0 ssh
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874724] Out of memory: Kill process 31046 (php) score 553 or sacrifice child
Jul 16 11:02:21 ip-10-61-174-46 kernel: [1499436.874736] Killed process 31046 (php) total-vm:2371332kB, anon-rss:2120020kB, file-rss:0kB
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745309] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745313] php5-fpm cpuset=/ mems_allowed=0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745316] Pid: 6563, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745318] Call Trace:
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745325] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745327] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745329] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745332] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745335] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745339] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745341] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745344] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745347] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745350] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745353] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745355] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745358] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745361] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745364] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745368] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745372] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745375] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745377] Mem-Info:
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745378] Node 0 DMA per-cpu:
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745380] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745381] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745382] Node 0 DMA32 per-cpu:
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745384] CPU 0: hi: 186, btch: 31 usd: 171
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745385] CPU 1: hi: 186, btch: 31 usd: 58
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745388] active_anon:931053 inactive_anon:1396 isolated_anon:0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745389] active_file:17 inactive_file:14 isolated_file:23
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745389] unevictable:0 dirty:42 writeback:0 unstable:0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745390] free:4001 slab_reclaimable:2468 slab_unreclaimable:3129
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745391] mapped:1355 shmem:1408 pagetables:4154 bounce:0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745392] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745399] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745402] Node 0 DMA32 free:7668kB min:7848kB low:9808kB high:11772kB active_anon:3724212kB inactive_anon:5584kB active_file:68kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):92kB present:3862656kB mlocked:0kB dirty:168kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9872kB slab_unreclaimable:12516kB kernel_stack:976kB pagetables:16616kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1201 all_unreclaimable? yes
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745409] lowmem_reserve[]: 0 0 0 0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745411] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745418] Node 0 DMA32: 779*4kB 316*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7692kB
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745425] 1452 total pagecache pages
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745426] 0 pages in swap cache
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745427] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745428] Free swap = 0kB
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.745429] Total swap = 0kB
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750952] 985072 pages RAM
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750954] 24998 pages reserved
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750956] 4239 pages shared
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750957] 952678 pages non-shared
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750959] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750968] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750972] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750975] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750978] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750981] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750984] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750988] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750991] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750995] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.750998] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751001] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751005] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751008] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751011] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751014] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751017] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751020] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751023] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751026] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751030] [ 3502] 1012 3502 71991 7871 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751033] [ 3504] 1012 3504 68832 6236 0 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751036] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751039] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751042] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751045] [ 6563] 0 6563 89584 2515 0 0 0 php5-fpm
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751048] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751051] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751055] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751058] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751061] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751064] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751067] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751070] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751073] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751076] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751079] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751082] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751086] [29842] 108 29842 6818 72 0 0 0 cleanup
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751089] [29843] 108 29843 6795 75 0 0 0 trivial-rewrite
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751092] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751095] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751098] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751101] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751104] [29864] 108 29864 6792 69 1 0 0 pickup
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751107] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751110] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751113] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751116] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751119] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751122] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751125] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751128] [31254] 1012 31254 609669 546789 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751131] [31509] 1012 31509 295927 233052 0 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751134] [31589] 1012 31589 73163 10584 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751138] [31743] 1012 31743 1099 25 0 0 0 sh
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751141] [31744] 1012 31744 68424 5872 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751143] [31756] 1012 31756 1099 26 1 0 0 sh
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751146] [31757] 1012 31757 4738 66 0 0 0 git
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751149] [31758] 1012 31758 68476 5522 1 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751152] [31762] 1012 31762 67123 4573 0 0 0 php
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751155] [31766] 1012 31766 1099 25 0 0 0 sh
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751158] [31767] 1012 31767 10901 175 1 0 0 ssh
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751161] Out of memory: Kill process 31254 (php) score 570 or sacrifice child
Jul 16 11:02:34 ip-10-61-174-46 kernel: [1499449.751172] Killed process 31254 (php) total-vm:2438676kB, anon-rss:2187156kB, file-rss:0kB
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936148] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936152] php cpuset=/ mems_allowed=0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936156] Pid: 32021, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936158] Call Trace:
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936166] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936172] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936176] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936182] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936185] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936189] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936191] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936194] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936198] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936200] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936202] [<ffffffff8113b231>] ? __do_fault+0x1/0x550
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936205] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936208] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936210] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936213] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936217] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936219] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936222] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936225] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936226] Mem-Info:
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936228] Node 0 DMA per-cpu:
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936230] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936231] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936232] Node 0 DMA32 per-cpu:
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936235] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936237] active_anon:931410 inactive_anon:1396 isolated_anon:0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936238] active_file:0 inactive_file:0 isolated_file:0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936239] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936239] free:4045 slab_reclaimable:2407 slab_unreclaimable:3121
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936240] mapped:1374 shmem:1408 pagetables:4081 bounce:0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936248] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936251] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3725640kB inactive_anon:5584kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5496kB shmem:5632kB slab_reclaimable:9628kB slab_unreclaimable:12484kB kernel_stack:944kB pagetables:16324kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1885 all_unreclaimable? yes
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936258] lowmem_reserve[]: 0 0 0 0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936260] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936267] Node 0 DMA32: 729*4kB 367*8kB 3*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7980kB
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936274] 1441 total pagecache pages
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936275] 0 pages in swap cache
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936276] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936277] Free swap = 0kB
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.936278] Total swap = 0kB
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941729] 985072 pages RAM
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941731] 24998 pages reserved
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941732] 4228 pages shared
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941733] 952852 pages non-shared
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941735] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941744] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941748] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941751] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941754] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941758] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941761] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941764] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941768] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941772] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941775] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941778] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941781] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941784] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941787] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941791] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941794] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941797] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941800] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941803] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941807] [ 3502] 1012 3502 71991 7875 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941809] [ 3504] 1012 3504 68832 6235 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941813] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941816] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941819] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941822] [ 6563] 0 6563 89584 2514 0 0 0 php5-fpm
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941825] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941828] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941831] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941834] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941837] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941840] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941843] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941847] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941850] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941853] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941856] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941859] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941862] [29843] 108 29843 6795 75 0 0 0 trivial-rewrite
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941866] [29844] 108 29844 11107 160 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941869] [29848] 108 29848 11107 158 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941872] [29855] 108 29855 11107 158 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941875] [29860] 108 29860 11107 157 0 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941878] [29864] 108 29864 6792 75 1 0 0 pickup
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941881] [29865] 108 29865 11107 159 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941884] [29904] 108 29904 11108 160 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941887] [29905] 108 29905 6792 70 1 0 0 scache
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941890] [29908] 108 29908 11108 159 0 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941893] [29912] 108 29912 11108 159 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941896] [29941] 108 29941 11108 159 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941899] [29957] 108 29957 11108 159 1 0 0 smtp
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941902] [31509] 1012 31509 646540 583687 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941905] [31920] 1012 31920 248312 185463 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941908] [31963] 1012 31963 72005 9469 0 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941911] [32017] 1012 32017 72913 10360 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941914] [32019] 1012 32019 71693 9110 1 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941917] [32021] 1012 32021 71629 9054 0 0 0 php
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941920] Out of memory: Kill process 31509 (php) score 609 or sacrifice child
Jul 16 11:02:49 ip-10-61-174-46 kernel: [1499464.941931] Killed process 31509 (php) total-vm:2586160kB, anon-rss:2334748kB, file-rss:0kB
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836152] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836156] php cpuset=/ mems_allowed=0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836160] Pid: 31920, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836162] Call Trace:
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836170] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836173] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836176] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836180] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836184] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836189] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836192] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836196] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836200] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836203] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836207] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836210] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836213] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836217] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836220] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836224] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836227] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836229] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836233] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836236] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836238] Mem-Info:
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836240] Node 0 DMA per-cpu:
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836242] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836244] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836246] Node 0 DMA32 per-cpu:
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836248] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836249] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836253] active_anon:929819 inactive_anon:1396 isolated_anon:0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836254] active_file:0 inactive_file:32 isolated_file:0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836254] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836255] free:3751 slab_reclaimable:2384 slab_unreclaimable:3535
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836256] mapped:1355 shmem:1408 pagetables:4132 bounce:0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836258] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836267] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836271] Node 0 DMA32 free:6668kB min:7848kB low:9808kB high:11772kB active_anon:3719276kB inactive_anon:5584kB active_file:0kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9536kB slab_unreclaimable:14140kB kernel_stack:920kB pagetables:16528kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2484 all_unreclaimable? yes
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836279] lowmem_reserve[]: 0 0 0 0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836283] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836292] Node 0 DMA32: 765*4kB 146*8kB 37*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 6676kB
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836301] 1428 total pagecache pages
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836303] 0 pages in swap cache
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836305] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836306] Free swap = 0kB
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.836308] Total swap = 0kB
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841849] 985072 pages RAM
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841851] 24998 pages reserved
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841852] 4229 pages shared
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841853] 953180 pages non-shared
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841855] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841864] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841868] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841871] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841874] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841877] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841881] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841884] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841888] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841891] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841895] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841898] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841901] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841904] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841907] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841910] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841913] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841916] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841919] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841923] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841926] [ 3502] 1012 3502 72055 7882 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841929] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841932] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841935] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841938] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841942] [ 6563] 0 6563 89584 2498 1 0 0 php5-fpm
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841945] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841948] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841951] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841954] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841957] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841960] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841963] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841966] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841969] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841972] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841975] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841978] [ 5438] 1002 5438 143346 79914 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841982] [29864] 108 29864 6792 75 1 0 0 pickup
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841985] [29905] 108 29905 6792 70 0 0 0 scache
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841988] [31920] 1012 31920 748110 685285 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841991] [32273] 1012 32273 138396 69851 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841994] [32305] 1012 32305 72971 10432 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.841997] [32306] 1012 32306 71748 9228 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842000] [32329] 1012 32329 72103 9543 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842003] [32332] 1012 32332 71628 9060 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842006] [32397] 1012 32397 1099 25 1 0 0 sh
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842009] [32398] 1012 32398 68424 5867 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842013] [32402] 1012 32402 68770 6209 1 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842016] [32410] 1012 32410 1099 26 0 0 0 sh
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842019] [32411] 1012 32411 4651 57 1 0 0 git
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842022] [32412] 1012 32412 64083 1476 0 0 0 php
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842024] [32414] 1012 32414 1099 26 0 0 0 sh
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842027] [32415] 1012 32415 5065 66 1 0 0 git
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842030] Out of memory: Kill process 31920 (php) score 715 or sacrifice child
Jul 16 11:03:07 ip-10-61-174-46 kernel: [1499482.842042] Killed process 31920 (php) total-vm:2992440kB, anon-rss:2741140kB, file-rss:0kB
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920152] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920156] php cpuset=/ mems_allowed=0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920159] Pid: 351, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920161] Call Trace:
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920169] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920172] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920175] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920179] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920183] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920187] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920190] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920194] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920198] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920201] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920205] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920207] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920210] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920214] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920218] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920221] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920225] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920227] Mem-Info:
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920229] Node 0 DMA per-cpu:
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920230] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920232] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920233] Node 0 DMA32 per-cpu:
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920235] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920236] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920238] active_anon:931489 inactive_anon:1396 isolated_anon:0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920239] active_file:5 inactive_file:9 isolated_file:0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920240] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920241] free:4048 slab_reclaimable:2347 slab_unreclaimable:3132
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920241] mapped:1355 shmem:1408 pagetables:3958 bounce:0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920243] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920249] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920252] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3725956kB inactive_anon:5584kB active_file:20kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9388kB slab_unreclaimable:12528kB kernel_stack:896kB pagetables:15832kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1130 all_unreclaimable? yes
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920259] lowmem_reserve[]: 0 0 0 0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920261] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920268] Node 0 DMA32: 1046*4kB 237*8kB 1*16kB 2*32kB 1*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 8016kB
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920275] 1429 total pagecache pages
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920276] 0 pages in swap cache
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920277] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920278] Free swap = 0kB
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.920279] Total swap = 0kB
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926360] 985072 pages RAM
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926362] 24998 pages reserved
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926364] 4245 pages shared
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926365] 952827 pages non-shared
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926367] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926375] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926379] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926382] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926385] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926388] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926391] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926395] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926398] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926402] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926405] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926408] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926412] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926415] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926418] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926421] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926424] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926427] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926430] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926433] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926438] [ 3502] 1012 3502 72055 7892 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926440] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926443] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926447] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926450] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926453] [ 6563] 0 6563 89584 2531 0 0 0 php5-fpm
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926456] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926459] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926462] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926465] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926468] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926471] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926475] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926478] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926480] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926483] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926486] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926489] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926493] [29864] 108 29864 6792 75 1 0 0 pickup
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926496] [29905] 108 29905 6792 70 0 0 0 scache
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926499] [32273] 1012 32273 739922 677100 0 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926502] [32595] 1012 32595 73138 10535 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926505] [ 351] 1012 351 155890 92692 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926508] [ 357] 1012 357 70981 8448 0 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926511] [ 379] 1012 379 1099 25 0 0 0 sh
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926514] [ 380] 1012 380 68424 5869 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926517] [ 384] 1012 384 70910 8326 0 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926520] [ 392] 1012 392 1099 25 0 0 0 sh
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926523] [ 393] 1012 393 4768 91 0 0 0 git
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926526] [ 394] 1012 394 68476 5523 1 0 0 php
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926529] [ 400] 1012 400 1099 25 0 0 0 sh
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926532] [ 401] 1012 401 10901 177 0 0 0 ssh
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926535] Out of memory: Kill process 32273 (php) score 706 or sacrifice child
Jul 16 11:03:26 ip-10-61-174-46 kernel: [1499501.926546] Killed process 32273 (php) total-vm:2959688kB, anon-rss:2708400kB, file-rss:0kB
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065334] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065339] php cpuset=/ mems_allowed=0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065342] Pid: 755, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065344] Call Trace:
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065352] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065356] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065359] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065363] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065367] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065371] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065374] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065378] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065382] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065385] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065389] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065392] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065395] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065399] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065403] [<ffffffff81144d28>] ? do_mmap_pgoff+0x348/0x360
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065406] [<ffffffff81144e21>] ? sys_mmap_pgoff+0xe1/0x230
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065410] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065412] Mem-Info:
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065414] Node 0 DMA per-cpu:
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065416] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065418] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065420] Node 0 DMA32 per-cpu:
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065422] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065424] CPU 1: hi: 186, btch: 31 usd: 12
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065427] active_anon:930125 inactive_anon:1396 isolated_anon:0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065428] active_file:0 inactive_file:0 isolated_file:13
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065429] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065430] free:3955 slab_reclaimable:2613 slab_unreclaimable:3366
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065431] mapped:1379 shmem:1408 pagetables:3782 bounce:0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065433] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065441] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065445] Node 0 DMA32 free:7484kB min:7848kB low:9808kB high:11772kB active_anon:3720500kB inactive_anon:5584kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):152kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5632kB slab_reclaimable:10452kB slab_unreclaimable:13464kB kernel_stack:856kB pagetables:15128kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:25 all_unreclaimable? yes
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065453] lowmem_reserve[]: 0 0 0 0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065456] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065493] Node 0 DMA32: 1102*4kB 230*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 7544kB
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065502] 1445 total pagecache pages
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065503] 0 pages in swap cache
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065505] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065506] Free swap = 0kB
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.065508] Total swap = 0kB
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071593] 985072 pages RAM
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071595] 24998 pages reserved
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071597] 4240 pages shared
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071598] 953112 pages non-shared
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071600] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071608] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071612] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071615] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071618] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071622] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071625] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071628] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071632] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071635] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071638] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071642] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071645] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071648] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071651] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071654] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071658] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071661] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071664] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071667] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071670] [ 3502] 1012 3502 72055 7902 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071673] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071676] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071679] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071682] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071685] [ 6563] 0 6563 89584 2503 1 0 0 php5-fpm
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071689] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071692] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071695] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071698] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071701] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071704] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071707] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071710] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071713] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071716] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071719] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071722] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071726] [29864] 108 29864 6792 75 1 0 0 pickup
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071729] [29905] 108 29905 6792 70 0 0 0 scache
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071732] [ 351] 1012 351 754061 691241 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071735] [ 667] 1012 667 73041 10481 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071738] [ 722] 1012 722 149755 86654 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071741] [ 740] 1012 740 70981 8379 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071744] [ 752] 1012 752 68245 5662 1 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071747] [ 754] 1012 754 1099 24 0 0 0 sh
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071750] [ 755] 1012 755 67613 5051 0 0 0 php
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071753] Out of memory: Kill process 351 (php) score 721 or sacrifice child
Jul 16 11:03:43 ip-10-61-174-46 kernel: [1499519.071763] Killed process 351 (php) total-vm:3016244kB, anon-rss:2764964kB, file-rss:0kB
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831839] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831843] php cpuset=/ mems_allowed=0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831846] Pid: 5438, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831847] Call Trace:
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831854] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831856] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831859] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831862] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831865] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831868] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831871] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831874] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831878] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831881] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831883] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831886] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831889] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831891] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831894] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831896] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831899] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831902] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831905] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831907] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831910] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831913] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831916] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831918] Mem-Info:
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831919] Node 0 DMA per-cpu:
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831921] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831922] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831923] Node 0 DMA32 per-cpu:
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831925] CPU 0: hi: 186, btch: 31 usd: 67
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831926] CPU 1: hi: 186, btch: 31 usd: 149
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831929] active_anon:932055 inactive_anon:1396 isolated_anon:0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831929] active_file:0 inactive_file:4 isolated_file:0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831930] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831931] free:4020 slab_reclaimable:2456 slab_unreclaimable:3023
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831932] mapped:1355 shmem:1408 pagetables:3444 bounce:0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831933] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831940] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831943] Node 0 DMA32 free:7744kB min:7848kB low:9808kB high:11772kB active_anon:3728220kB inactive_anon:5584kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:9824kB slab_unreclaimable:12092kB kernel_stack:824kB pagetables:13776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:305 all_unreclaimable? yes
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831949] lowmem_reserve[]: 0 0 0 0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831951] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831958] Node 0 DMA32: 1134*4kB 145*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7744kB
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831965] 1432 total pagecache pages
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831966] 0 pages in swap cache
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831967] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831968] Free swap = 0kB
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.831969] Total swap = 0kB
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837469] 985072 pages RAM
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837471] 24998 pages reserved
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837472] 4248 pages shared
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837474] 952693 pages non-shared
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837475] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837484] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837488] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837491] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837494] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837497] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837500] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837504] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837507] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837511] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837514] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837517] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837521] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837524] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837527] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837530] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837533] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837536] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837540] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837543] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837546] [ 3502] 1012 3502 72055 7916 1 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837549] [ 3504] 1012 3504 68832 6235 1 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837552] [ 3506] 1012 3506 69239 6591 1 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837555] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837559] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837562] [ 6563] 0 6563 89584 2521 0 0 0 php5-fpm
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837565] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837568] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837571] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837574] [11050] 1002 11050 18359 201 1 0 0 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837577] [11051] 1002 11051 6542 1750 1 0 0 bash
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837580] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837583] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837586] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837589] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837592] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837595] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837598] [ 5438] 1002 5438 143346 79914 0 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837602] [29864] 108 29864 6792 75 1 0 0 pickup
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837605] [29905] 108 29905 6792 70 0 0 0 scache
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837608] [ 722] 1012 722 866257 803438 0 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837611] [ 1209] 1012 1209 68513 5969 1 0 0 php
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837614] Out of memory: Kill process 722 (php) score 838 or sacrifice child
Jul 16 11:04:04 ip-10-61-174-46 kernel: [1499539.837625] Killed process 722 (php) total-vm:3465028kB, anon-rss:3213752kB, file-rss:0kB
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368090] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368095] php cpuset=/ mems_allowed=0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368097] Pid: 32580, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368099] Call Trace:
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368106] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368108] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368110] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368113] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368116] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368120] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368122] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368125] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368128] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368130] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368134] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368136] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368138] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368140] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368142] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368146] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368149] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368152] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368154] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368158] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368160] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368162] Mem-Info:
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368163] Node 0 DMA per-cpu:
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368165] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368166] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368167] Node 0 DMA32 per-cpu:
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368169] CPU 0: hi: 186, btch: 31 usd: 93
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368170] CPU 1: hi: 186, btch: 31 usd: 38
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368173] active_anon:928534 inactive_anon:1396 isolated_anon:0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368174] active_file:0 inactive_file:18 isolated_file:0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368174] unevictable:0 dirty:2 writeback:0 unstable:0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368175] free:4046 slab_reclaimable:2697 slab_unreclaimable:4320
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368176] mapped:1355 shmem:1408 pagetables:3990 bounce:0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368177] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368184] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368187] Node 0 DMA32 free:7848kB min:7848kB low:9808kB high:11772kB active_anon:3714136kB inactive_anon:5584kB active_file:0kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:8kB writeback:0kB mapped:5420kB shmem:5632kB slab_reclaimable:10788kB slab_unreclaimable:17280kB kernel_stack:864kB pagetables:15960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:228 all_unreclaimable? yes
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368193] lowmem_reserve[]: 0 0 0 0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368196] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368202] Node 0 DMA32: 1478*4kB 42*8kB 6*16kB 1*32kB 3*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 7848kB
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368209] 1440 total pagecache pages
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368210] 0 pages in swap cache
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368211] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368212] Free swap = 0kB
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.368213] Total swap = 0kB
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373768] 985072 pages RAM
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373770] 24998 pages reserved
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373772] 4236 pages shared
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373773] 952758 pages non-shared
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373775] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373783] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373787] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373790] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373793] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373796] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373800] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373803] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373807] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373810] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373813] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373817] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373820] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373823] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373826] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373829] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373832] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373835] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373838] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373842] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373845] [ 3502] 1012 3502 77751 13618 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373848] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373851] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373854] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373857] [ 3533] 1012 3533 68271 5639 0 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373860] [ 6563] 0 6563 89584 2505 1 0 0 php5-fpm
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373863] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373866] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373869] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373872] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373875] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373879] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373882] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373885] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373888] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373891] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373894] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373897] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373901] [32569] 1012 32569 385716 322788 0 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373904] [32580] 1012 32580 508284 445418 0 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373907] [32646] 1012 32646 71045 8523 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373910] [ 367] 1012 367 141517 75169 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373913] [ 394] 1012 394 71045 8452 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373916] [ 425] 1012 425 70724 8165 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373919] [ 432] 1012 432 70910 8367 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373922] [ 457] 1012 457 1099 24 1 0 0 sh
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373925] [ 458] 1012 458 1099 25 1 0 0 sh
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373928] [ 459] 1012 459 63812 1577 1 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373931] [ 460] 1012 460 63812 1577 0 0 0 php
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373934] Out of memory: Kill process 32580 (php) score 464 or sacrifice child
Jul 16 13:17:25 ip-10-61-174-46 kernel: [1507541.373945] Killed process 32580 (php) total-vm:2033136kB, anon-rss:1781668kB, file-rss:4kB
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162741] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162746] php cpuset=/ mems_allowed=0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162750] Pid: 3526, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162752] Call Trace:
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162760] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162763] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162766] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162770] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162774] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162779] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162781] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162784] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162787] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162790] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162793] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162795] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162798] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162800] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162804] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162808] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162812] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162816] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162819] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162821] Mem-Info:
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162823] Node 0 DMA per-cpu:
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162825] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162827] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162828] Node 0 DMA32 per-cpu:
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162830] CPU 0: hi: 186, btch: 31 usd: 72
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162832] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162836] active_anon:931116 inactive_anon:1396 isolated_anon:0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162837] active_file:58 inactive_file:86 isolated_file:0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162838] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162838] free:4020 slab_reclaimable:2609 slab_unreclaimable:3744
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162839] mapped:1462 shmem:1408 pagetables:3471 bounce:0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162841] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162850] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162854] Node 0 DMA32 free:7744kB min:7848kB low:9808kB high:11772kB active_anon:3724464kB inactive_anon:5584kB active_file:232kB inactive_file:344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5848kB shmem:5632kB slab_reclaimable:10436kB slab_unreclaimable:14976kB kernel_stack:824kB pagetables:13884kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:953 all_unreclaimable? yes
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162862] lowmem_reserve[]: 0 0 0 0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162865] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162875] Node 0 DMA32: 1424*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7744kB
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162884] 1608 total pagecache pages
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162885] 0 pages in swap cache
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162887] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162889] Free swap = 0kB
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.162890] Total swap = 0kB
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170200] 985072 pages RAM
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170203] 24998 pages reserved
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170204] 4485 pages shared
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170205] 952672 pages non-shared
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170207] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170216] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170219] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170222] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170225] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170228] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170231] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170233] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170236] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170240] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170243] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170247] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170250] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170253] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170256] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170258] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170261] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170264] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170266] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170270] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170273] [ 3502] 1012 3502 77751 13634 0 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170275] [ 3504] 1012 3504 68832 6247 1 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170278] [ 3506] 1012 3506 69239 6616 1 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170281] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170283] [ 3533] 1012 3533 68271 5639 0 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170286] [ 6563] 0 6563 89584 2491 1 0 0 php5-fpm
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170289] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170292] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170294] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170297] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170300] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170302] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170305] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170308] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170311] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170314] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170316] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170319] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170322] [32569] 1012 32569 596032 533187 0 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170325] [ 367] 1012 367 395764 332951 0 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170327] [ 589] 1012 589 71045 8507 0 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170330] [ 592] 1012 592 70788 8232 0 0 0 php
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170332] Out of memory: Kill process 32569 (php) score 556 or sacrifice child
Jul 16 13:17:35 ip-10-61-174-46 kernel: [1507551.170344] Killed process 32569 (php) total-vm:2384128kB, anon-rss:2132644kB, file-rss:104kB
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860289] sh invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860293] sh cpuset=/ mems_allowed=0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860296] Pid: 842, comm: sh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860297] Call Trace:
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860304] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860306] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860309] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860312] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860315] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860319] [<ffffffff811894c3>] ? user_path_at_empty+0x63/0xa0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860321] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860324] [<ffffffff81196e8e>] ? vfsmount_lock_local_unlock+0x1e/0x30
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860326] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860328] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860331] [<ffffffff8117e6aa>] ? sys_newstat+0x2a/0x40
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860335] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860339] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860341] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860343] Mem-Info:
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860344] Node 0 DMA per-cpu:
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860346] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860347] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860348] Node 0 DMA32 per-cpu:
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860350] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860351] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860354] active_anon:931138 inactive_anon:1396 isolated_anon:0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860355] active_file:0 inactive_file:8 isolated_file:0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860355] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860356] free:4097 slab_reclaimable:2472 slab_unreclaimable:3745
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860357] mapped:1363 shmem:1408 pagetables:3706 bounce:0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860358] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860365] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860368] Node 0 DMA32 free:8052kB min:7848kB low:9808kB high:11772kB active_anon:3724552kB inactive_anon:5584kB active_file:0kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5632kB slab_reclaimable:9888kB slab_unreclaimable:14980kB kernel_stack:840kB pagetables:14824kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:54 all_unreclaimable? yes
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860374] lowmem_reserve[]: 0 0 0 0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860377] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860384] Node 0 DMA32: 1437*4kB 20*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8084kB
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860390] 1473 total pagecache pages
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860391] 0 pages in swap cache
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860393] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860394] Free swap = 0kB
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.860395] Total swap = 0kB
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865725] 985072 pages RAM
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865728] 24998 pages reserved
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865729] 4231 pages shared
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865730] 952812 pages non-shared
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865732] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865740] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865744] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865747] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865750] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865753] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865756] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865760] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865763] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865767] [ 790] 101 790 63477 658 0 0 0 rsyslogd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865770] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865773] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865776] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865779] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865782] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865785] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865788] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865792] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865795] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865798] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865801] [ 3502] 1012 3502 77751 13628 1 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865804] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865807] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865810] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865813] [ 3533] 1012 3533 68271 5639 0 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865817] [ 6563] 0 6563 89584 2519 0 0 0 php5-fpm
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865820] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865823] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865826] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865829] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865832] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865835] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865839] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865842] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865845] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865848] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865851] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865854] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865858] [ 367] 1012 367 806541 743686 0 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865861] [ 749] 1012 749 180370 109747 0 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865864] [ 754] 1012 754 71045 8451 0 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865867] [ 759] 1012 759 71043 8455 0 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865870] [ 819] 1012 819 70981 8427 1 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865873] [ 836] 1012 836 65960 3906 1 0 0 php
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865876] [ 842] 1012 842 1099 27 1 0 0 sh
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865879] Out of memory: Kill process 367 (php) score 776 or sacrifice child
Jul 16 13:17:47 ip-10-61-174-46 kernel: [1507562.865889] Killed process 367 (php) total-vm:3226164kB, anon-rss:2974744kB, file-rss:0kB
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148127] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148131] php cpuset=/ mems_allowed=0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148134] Pid: 749, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148136] Call Trace:
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148142] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148145] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148147] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148150] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148153] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148157] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148159] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148162] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148165] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148168] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148171] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148173] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148175] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148177] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148181] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148183] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148187] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148189] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148191] Mem-Info:
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148192] Node 0 DMA per-cpu:
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148194] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148195] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148196] Node 0 DMA32 per-cpu:
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148198] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148199] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148202] active_anon:931403 inactive_anon:1396 isolated_anon:0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148202] active_file:0 inactive_file:0 isolated_file:0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148203] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148204] free:4048 slab_reclaimable:2601 slab_unreclaimable:3474
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148204] mapped:1387 shmem:1408 pagetables:3593 bounce:0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148206] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148213] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148216] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3725612kB inactive_anon:5584kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5548kB shmem:5632kB slab_reclaimable:10404kB slab_unreclaimable:13896kB kernel_stack:840kB pagetables:14372kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148222] lowmem_reserve[]: 0 0 0 0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148225] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148232] Node 0 DMA32: 1437*4kB 5*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148238] 1421 total pagecache pages
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148239] 0 pages in swap cache
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148241] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148242] Free swap = 0kB
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.148243] Total swap = 0kB
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153421] 985072 pages RAM
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153423] 24998 pages reserved
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153424] 4318 pages shared
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153425] 952853 pages non-shared
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153427] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153436] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153440] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153443] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153446] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153449] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153453] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153456] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153459] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153463] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153466] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153469] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153473] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153476] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153479] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153482] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153485] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153488] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153491] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153495] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153498] [ 3502] 1012 3502 77751 13638 1 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153501] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153504] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153507] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153510] [ 3533] 1012 3533 68271 5639 0 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153513] [ 6563] 0 6563 89584 2503 1 0 0 php5-fpm
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153517] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153520] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153523] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153526] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153529] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153532] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153535] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153538] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153541] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153545] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153547] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153551] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153554] [ 749] 1012 749 673670 610828 0 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153557] [ 1014] 1012 1014 71045 8513 1 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153560] [ 1019] 1012 1019 312303 249395 0 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153563] [ 1312] 1012 1312 70908 8352 0 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153566] [ 1316] 1012 1316 1099 24 1 0 0 sh
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153569] [ 1317] 1012 1317 68424 5868 0 0 0 php
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153572] [ 1331] 1012 1331 1099 25 0 0 0 sh
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153575] Out of memory: Kill process 749 (php) score 637 or sacrifice child
Jul 16 13:18:05 ip-10-61-174-46 kernel: [1507581.153586] Killed process 749 (php) total-vm:2694680kB, anon-rss:2443204kB, file-rss:108kB
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116027] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116032] ntpd cpuset=/ mems_allowed=0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116035] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116037] Call Trace:
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116046] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116049] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116065] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116068] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116072] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116077] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116080] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116084] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116087] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116091] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116094] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116097] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116101] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116105] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116110] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116112] Mem-Info:
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116114] Node 0 DMA per-cpu:
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116116] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116118] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116119] Node 0 DMA32 per-cpu:
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116122] CPU 0: hi: 186, btch: 31 usd: 118
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116124] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116127] active_anon:931206 inactive_anon:1396 isolated_anon:0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116128] active_file:6 inactive_file:5 isolated_file:22
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116129] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116129] free:4032 slab_reclaimable:2650 slab_unreclaimable:3356
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116130] mapped:1352 shmem:1408 pagetables:3702 bounce:0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116132] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116141] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116144] Node 0 DMA32 free:7792kB min:7848kB low:9808kB high:11772kB active_anon:3724824kB inactive_anon:5584kB active_file:24kB inactive_file:20kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5632kB slab_reclaimable:10600kB slab_unreclaimable:13424kB kernel_stack:848kB pagetables:14808kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:234 all_unreclaimable? yes
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116153] lowmem_reserve[]: 0 0 0 0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116156] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116166] Node 0 DMA32: 1443*4kB 0*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7884kB
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116175] 1427 total pagecache pages
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116176] 0 pages in swap cache
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116178] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116179] Free swap = 0kB
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.116180] Total swap = 0kB
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121501] 985072 pages RAM
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121503] 24998 pages reserved
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121504] 4237 pages shared
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121505] 952749 pages non-shared
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121507] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121514] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121516] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121519] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121521] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121523] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121525] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121528] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121530] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121533] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121535] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121537] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121540] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121542] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121544] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121547] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121549] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121551] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121553] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121556] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121558] [ 3502] 1012 3502 77815 13642 1 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121560] [ 3504] 1012 3504 68832 6235 1 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121563] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121565] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121567] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121569] [ 6563] 0 6563 89584 2487 1 0 0 php5-fpm
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121572] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121574] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121576] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121578] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121580] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121582] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121585] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121587] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121589] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121591] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121593] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121596] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121598] [ 1014] 1012 1014 357234 294351 0 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121600] [ 1019] 1012 1019 622274 559436 0 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121603] [ 1388] 1012 1388 71043 8514 0 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121605] [ 1552] 1012 1552 70314 7758 1 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121607] [ 1556] 1012 1556 69860 7319 1 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121609] [ 1565] 1012 1565 67912 5345 1 0 0 php
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121611] [ 1577] 1012 1577 1099 26 0 0 0 sh
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121613] [ 1578] 1012 1578 276172 69 0 0 0 git
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121615] Out of memory: Kill process 1019 (php) score 583 or sacrifice child
Jul 16 13:18:18 ip-10-61-174-46 kernel: [1507594.121626] Killed process 1019 (php) total-vm:2489096kB, anon-rss:2237744kB, file-rss:0kB
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904138] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904143] php cpuset=/ mems_allowed=0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904146] Pid: 1649, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904148] Call Trace:
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904156] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904159] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904172] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904175] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904183] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904187] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904191] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904195] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904198] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904202] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904205] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904208] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904212] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904215] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904218] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904221] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904224] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904227] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904230] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904233] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904234] Mem-Info:
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904236] Node 0 DMA per-cpu:
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904238] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904239] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904240] Node 0 DMA32 per-cpu:
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904242] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904243] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904246] active_anon:931412 inactive_anon:1396 isolated_anon:0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904246] active_file:14 inactive_file:25 isolated_file:0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904247] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904248] free:4048 slab_reclaimable:2600 slab_unreclaimable:3354
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904248] mapped:1364 shmem:1408 pagetables:3754 bounce:0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904250] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904256] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904259] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3725648kB inactive_anon:5584kB active_file:56kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5456kB shmem:5632kB slab_reclaimable:10400kB slab_unreclaimable:13416kB kernel_stack:872kB pagetables:15016kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:323 all_unreclaimable? yes
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904266] lowmem_reserve[]: 0 0 0 0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904268] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904275] Node 0 DMA32: 1484*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904282] 1433 total pagecache pages
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904283] 0 pages in swap cache
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904284] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904285] Free swap = 0kB
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.904286] Total swap = 0kB
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909625] 985072 pages RAM
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909628] 24998 pages reserved
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909629] 4229 pages shared
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909630] 952859 pages non-shared
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909632] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909640] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909644] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909648] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909651] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909654] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909657] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909660] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909664] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909668] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909671] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909674] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909677] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909680] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909683] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909687] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909690] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909693] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909696] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909699] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909702] [ 3502] 1012 3502 77815 13648 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909705] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909708] [ 3506] 1012 3506 69239 6606 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909711] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909714] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909717] [ 6563] 0 6563 89584 2526 0 0 0 php5-fpm
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909720] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909723] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909727] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909730] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909733] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909736] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909739] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909742] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909745] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909748] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909751] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909754] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909758] [ 1014] 1012 1014 653316 590440 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909761] [ 1565] 1012 1565 326768 263938 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909763] [ 1649] 1012 1649 71045 8466 1 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909766] [ 1706] 1012 1706 71045 8452 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909769] [ 1738] 1012 1738 1099 25 1 0 0 sh
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909772] [ 1740] 1012 1740 68424 5867 0 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909775] [ 1757] 1012 1757 1099 26 0 0 0 sh
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909778] [ 1758] 1012 1758 14084 63 0 0 0 git
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909781] [ 1759] 1012 1759 68476 5522 1 0 0 php
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909784] [ 1771] 1012 1771 1099 25 0 0 0 sh
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909787] [ 1772] 1012 1772 10901 175 0 0 0 ssh
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909790] Out of memory: Kill process 1014 (php) score 616 or sacrifice child
Jul 16 13:18:32 ip-10-61-174-46 kernel: [1507607.909800] Killed process 1014 (php) total-vm:2613264kB, anon-rss:2361760kB, file-rss:0kB
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817779] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817783] php cpuset=/ mems_allowed=0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817786] Pid: 1565, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817788] Call Trace:
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817795] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817797] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817800] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817802] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817807] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817810] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817814] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817816] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817819] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817822] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817824] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817828] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817830] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817832] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817835] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817837] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817840] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817843] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817846] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817849] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817851] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817853] Mem-Info:
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817854] Node 0 DMA per-cpu:
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817856] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817858] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817859] Node 0 DMA32 per-cpu:
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817861] CPU 0: hi: 186, btch: 31 usd: 31
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817862] CPU 1: hi: 186, btch: 31 usd: 151
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817864] active_anon:931825 inactive_anon:1396 isolated_anon:0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817865] active_file:6 inactive_file:16 isolated_file:22
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817866] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817866] free:4027 slab_reclaimable:2438 slab_unreclaimable:3192
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817867] mapped:1352 shmem:1408 pagetables:3532 bounce:0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817869] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817875] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817878] Node 0 DMA32 free:7772kB min:7848kB low:9808kB high:11772kB active_anon:3727300kB inactive_anon:5584kB active_file:24kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5632kB slab_reclaimable:9752kB slab_unreclaimable:12768kB kernel_stack:784kB pagetables:14128kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:323 all_unreclaimable? yes
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817884] lowmem_reserve[]: 0 0 0 0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817887] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817894] Node 0 DMA32: 1431*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7772kB
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817900] 1456 total pagecache pages
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817902] 0 pages in swap cache
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817903] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817904] Free swap = 0kB
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.817905] Total swap = 0kB
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823089] 985072 pages RAM
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823092] 24998 pages reserved
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823094] 4234 pages shared
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823095] 952731 pages non-shared
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823097] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823105] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823109] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823112] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823115] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823118] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823122] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823125] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823128] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823132] [ 790] 101 790 63477 658 1 0 0 rsyslogd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823135] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823139] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823142] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823145] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823148] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823151] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823154] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823157] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823161] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823164] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823167] [ 3502] 1012 3502 77815 13654 0 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823170] [ 3504] 1012 3504 68832 6235 1 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823173] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823176] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823179] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823182] [ 6563] 0 6563 89584 2510 0 0 0 php5-fpm
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823186] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823189] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823192] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823194] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823197] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823201] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823204] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823207] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823210] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823213] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823216] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823219] [12591] 108 12591 6792 71 0 0 0 pickup
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823223] [ 1565] 1012 1565 707528 644696 0 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823226] [ 1823] 1012 1823 284653 221791 1 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823229] [ 1915] 1012 1915 71045 8454 1 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823232] [ 1929] 1012 1929 71043 8456 0 0 0 php
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823235] [ 1979] 1012 1979 1099 25 1 0 0 sh
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823238] Out of memory: Kill process 1565 (php) score 672 or sacrifice child
Jul 16 13:18:45 ip-10-61-174-46 kernel: [1507620.823249] Killed process 1565 (php) total-vm:2830112kB, anon-rss:2578776kB, file-rss:8kB
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816182] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816187] php cpuset=/ mems_allowed=0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816190] Pid: 1823, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816192] Call Trace:
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816200] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816203] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816206] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816210] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816214] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816219] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816222] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816226] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816230] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816233] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816237] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816240] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816243] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816247] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816249] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816254] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816258] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816262] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816266] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816269] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816272] Mem-Info:
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816273] Node 0 DMA per-cpu:
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816276] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816278] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816279] Node 0 DMA32 per-cpu:
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816281] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816283] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816287] active_anon:930949 inactive_anon:1396 isolated_anon:0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816288] active_file:0 inactive_file:31 isolated_file:0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816289] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816290] free:2826 slab_reclaimable:2650 slab_unreclaimable:3515
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816290] mapped:1359 shmem:1408 pagetables:3914 bounce:0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816293] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816301] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816305] Node 0 DMA32 free:2968kB min:7848kB low:9808kB high:11772kB active_anon:3723796kB inactive_anon:5584kB active_file:0kB inactive_file:124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5436kB shmem:5632kB slab_reclaimable:10600kB slab_unreclaimable:14060kB kernel_stack:912kB pagetables:15656kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:31 all_unreclaimable? yes
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816313] lowmem_reserve[]: 0 0 0 0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816317] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816327] Node 0 DMA32: 388*4kB 25*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 2968kB
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816336] 1439 total pagecache pages
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816338] 0 pages in swap cache
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816339] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816341] Free swap = 0kB
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.816342] Total swap = 0kB
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821742] 985072 pages RAM
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821744] 24998 pages reserved
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821746] 4238 pages shared
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821747] 954097 pages non-shared
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821749] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821757] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821760] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821764] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821767] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821770] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821773] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821776] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821780] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821784] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821787] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821790] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821794] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821797] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821800] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821803] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821806] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821809] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821812] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821815] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821819] [ 3502] 1012 3502 77815 13661 0 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821822] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821825] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821827] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821830] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821834] [ 6563] 0 6563 89584 2534 0 0 0 php5-fpm
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821837] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821840] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821843] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821846] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821849] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821852] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821856] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821859] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821862] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821865] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821868] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821871] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821874] [ 1823] 1012 1823 833615 770813 0 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821877] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821881] [ 2143] 108 2143 6795 71 0 0 0 trivial-rewrite
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821884] [ 2144] 108 2144 11107 159 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821887] [ 2148] 108 2148 11107 158 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821890] [ 2152] 108 2152 6792 69 1 0 0 scache
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821893] [ 2156] 108 2156 11108 159 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821896] [ 2160] 108 2160 11108 159 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821899] [ 2164] 108 2164 11108 159 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821902] [ 2168] 108 2168 11108 159 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821905] [ 2178] 108 2178 11108 161 1 0 0 smtp
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821908] [ 2237] 1012 2237 71045 8484 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821911] [ 2294] 1012 2294 146093 81764 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821914] [ 2441] 1012 2441 70910 8371 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821917] [ 2449] 1012 2449 57180 6818 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821920] [ 2453] 1012 2453 67463 4899 1 0 0 php
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821923] Out of memory: Kill process 1823 (php) score 804 or sacrifice child
Jul 16 13:19:05 ip-10-61-174-46 kernel: [1507640.821934] Killed process 1823 (php) total-vm:3334460kB, anon-rss:3083252kB, file-rss:0kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448147] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448152] php cpuset=/ mems_allowed=0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448155] Pid: 2592, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448158] Call Trace:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448166] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448171] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448176] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448179] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448182] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448185] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448188] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448192] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448198] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448202] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448205] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448208] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448211] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448214] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448216] Mem-Info:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448217] Node 0 DMA per-cpu:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448219] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448220] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448221] Node 0 DMA32 per-cpu:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448223] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448224] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448227] active_anon:930011 inactive_anon:1396 isolated_anon:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448228] active_file:7 inactive_file:23 isolated_file:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448228] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448229] free:3236 slab_reclaimable:2547 slab_unreclaimable:3622
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448230] mapped:1350 shmem:1408 pagetables:4114 bounce:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448231] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448238] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448241] Node 0 DMA32 free:4608kB min:7848kB low:9808kB high:11772kB active_anon:3720044kB inactive_anon:5584kB active_file:28kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5632kB slab_reclaimable:10188kB slab_unreclaimable:14488kB kernel_stack:920kB pagetables:16456kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5729 all_unreclaimable? yes
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448247] lowmem_reserve[]: 0 0 0 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448250] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448257] Node 0 DMA32: 792*4kB 23*8kB 0*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 4632kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448263] 1456 total pagecache pages
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448264] 0 pages in swap cache
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448266] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448267] Free swap = 0kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.448268] Total swap = 0kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453653] 985072 pages RAM
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453656] 24998 pages reserved
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453657] 4237 pages shared
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453658] 953681 pages non-shared
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453660] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453669] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453672] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453676] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453678] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453682] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453685] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453689] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453692] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453696] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453699] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453702] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453706] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453709] [ 983] 108 983 6834 107 1 0 0 qmgr
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453712] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453715] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453718] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453721] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453724] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453728] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453731] [ 3502] 1012 3502 77815 13665 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453734] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453737] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453740] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453743] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453746] [ 6563] 0 6563 89584 2518 1 0 0 php5-fpm
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453750] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453753] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453756] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453759] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453762] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453765] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453768] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453771] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453774] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453777] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453780] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453783] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453787] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453790] [ 2143] 108 2143 6795 71 1 0 0 trivial-rewrite
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453793] [ 2144] 108 2144 11107 159 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453796] [ 2148] 108 2148 11107 158 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453799] [ 2152] 108 2152 6792 69 1 0 0 scache
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453802] [ 2156] 108 2156 11108 159 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453805] [ 2160] 108 2160 11108 159 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453808] [ 2164] 108 2164 11108 159 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453811] [ 2168] 108 2168 11108 159 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453814] [ 2178] 108 2178 11108 161 1 0 0 smtp
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453817] [ 2294] 1012 2294 650052 587210 0 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453820] [ 2460] 1012 2460 71045 8511 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453823] [ 2560] 1012 2560 264243 201351 0 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453826] [ 2584] 1012 2584 70981 8447 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453829] [ 2585] 1012 2585 123578 54235 0 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453832] [ 2592] 1012 2592 70970 8420 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453835] [ 2599] 1012 2599 70981 8425 1 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453838] [ 2646] 1012 2646 1099 25 0 0 0 sh
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453841] [ 2647] 1012 2647 65627 3569 0 0 0 php
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453843] Out of memory: Kill process 2294 (php) score 612 or sacrifice child
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.453855] Killed process 2294 (php) total-vm:2600208kB, anon-rss:2348840kB, file-rss:0kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472088] php: page allocation failure: order:0, mode:0x201da
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472094] Pid: 2294, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472096] Call Trace:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472104] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472109] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472113] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472117] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472121] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472126] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472129] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472133] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472138] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472143] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472147] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472150] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472154] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472156] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472159] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472162] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472166] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472171] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472175] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472178] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472180] Mem-Info:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472182] Node 0 DMA per-cpu:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472184] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472188] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472189] Node 0 DMA32 per-cpu:
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472192] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472193] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472197] active_anon:930011 inactive_anon:1396 isolated_anon:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472198] active_file:7 inactive_file:23 isolated_file:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472199] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472200] free:3205 slab_reclaimable:2547 slab_unreclaimable:3622
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472201] mapped:1350 shmem:1408 pagetables:4114 bounce:0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472203] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472211] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472215] Node 0 DMA32 free:4484kB min:7848kB low:9808kB high:11772kB active_anon:3720044kB inactive_anon:5584kB active_file:28kB inactive_file:92kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5400kB shmem:5632kB slab_reclaimable:10188kB slab_unreclaimable:14488kB kernel_stack:920kB pagetables:16456kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:21 all_unreclaimable? no
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472223] lowmem_reserve[]: 0 0 0 0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472227] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472238] Node 0 DMA32: 791*4kB 12*8kB 0*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 4540kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472248] 1456 total pagecache pages
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472249] 0 pages in swap cache
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472251] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472252] Free swap = 0kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.472254] Total swap = 0kB
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.479353] 985072 pages RAM
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.479355] 24998 pages reserved
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.479356] 4237 pages shared
Jul 16 13:19:18 ip-10-61-174-46 kernel: [1507654.479357] 953704 pages non-shared
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621467] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621471] php cpuset=/ mems_allowed=0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621474] Pid: 2961, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621476] Call Trace:
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621483] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621485] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621487] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621490] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621493] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621497] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621499] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621502] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621504] [<ffffffff8113ca82>] ? do_wp_page+0x382/0x740
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621507] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621510] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621512] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621514] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621517] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621520] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621524] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621527] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621530] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621533] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621535] Mem-Info:
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621536] Node 0 DMA per-cpu:
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621538] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621539] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621540] Node 0 DMA32 per-cpu:
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621542] CPU 0: hi: 186, btch: 31 usd: 68
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621543] CPU 1: hi: 186, btch: 31 usd: 99
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621546] active_anon:930450 inactive_anon:1396 isolated_anon:0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621547] active_file:22 inactive_file:7 isolated_file:22
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621547] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621548] free:4033 slab_reclaimable:2505 slab_unreclaimable:3279
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621549] mapped:1386 shmem:1408 pagetables:4404 bounce:0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621550] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621557] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621560] Node 0 DMA32 free:7796kB min:7848kB low:9808kB high:11772kB active_anon:3721800kB inactive_anon:5584kB active_file:88kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5544kB shmem:5632kB slab_reclaimable:10020kB slab_unreclaimable:13116kB kernel_stack:1016kB pagetables:17616kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:307 all_unreclaimable? yes
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621566] lowmem_reserve[]: 0 0 0 0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621569] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621575] Node 0 DMA32: 1179*4kB 123*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7796kB
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621582] 1434 total pagecache pages
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621583] 0 pages in swap cache
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621585] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621586] Free swap = 0kB
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.621587] Total swap = 0kB
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626819] 985072 pages RAM
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626822] 24998 pages reserved
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626823] 4234 pages shared
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626824] 952716 pages non-shared
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626826] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626835] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626838] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626841] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626845] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626848] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626851] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626855] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626858] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626862] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626865] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626868] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626871] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626874] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626878] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626881] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626884] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626887] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626890] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626894] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626897] [ 3502] 1012 3502 77815 13671 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626900] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626903] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626906] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626909] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626912] [ 6563] 0 6563 89584 2502 1 0 0 php5-fpm
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626916] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626919] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626922] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626925] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626928] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626931] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626934] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626937] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626942] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626945] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626948] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626951] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626955] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626958] [ 2143] 108 2143 6795 71 0 0 0 trivial-rewrite
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626961] [ 2144] 108 2144 11107 159 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626965] [ 2148] 108 2148 11107 158 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626967] [ 2152] 108 2152 6792 69 1 0 0 scache
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626970] [ 2156] 108 2156 11108 159 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626973] [ 2160] 108 2160 11108 159 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626976] [ 2164] 108 2164 11108 159 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626980] [ 2168] 108 2168 11108 159 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626983] [ 2178] 108 2178 11108 161 1 0 0 smtp
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626986] [ 2560] 1012 2560 513916 451085 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626989] [ 2585] 1012 2585 441785 378916 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626992] [ 2941] 1012 2941 1099 24 0 0 0 sh
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626995] [ 2944] 1012 2944 68424 5873 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.626999] [ 2956] 1012 2956 1099 25 1 0 0 sh
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627002] [ 2961] 1012 2961 68424 5871 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627004] [ 3008] 1012 3008 70723 8129 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627008] [ 3027] 1012 3027 1099 26 0 0 0 sh
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627011] [ 3028] 1012 3028 4825 67 1 0 0 git
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627014] [ 3032] 1012 3032 68142 5151 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627017] [ 3042] 1012 3042 1099 24 1 0 0 sh
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627019] [ 3045] 1012 3045 67610 5085 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627022] [ 3048] 1012 3048 1099 26 1 0 0 sh
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627025] [ 3052] 1012 3052 4669 59 0 0 0 git
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627028] [ 3055] 1012 3055 66435 3981 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627031] [ 3061] 1012 3061 69733 7159 0 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627035] [ 3062] 1012 3062 1099 25 1 0 0 sh
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627038] [ 3063] 1012 3063 64782 2720 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627041] [ 3067] 1012 3067 69026 6455 1 0 0 php
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627044] Out of memory: Kill process 2560 (php) score 470 or sacrifice child
Jul 16 13:19:35 ip-10-61-174-46 kernel: [1507670.627055] Killed process 2560 (php) total-vm:2055664kB, anon-rss:1804340kB, file-rss:0kB
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252164] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252169] php cpuset=/ mems_allowed=0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252172] Pid: 3324, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252174] Call Trace:
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252182] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252185] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252187] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252190] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252196] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252198] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252202] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252204] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252207] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252210] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252213] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252215] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252218] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252220] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252223] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252226] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252228] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252230] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252233] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252235] Mem-Info:
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252236] Node 0 DMA per-cpu:
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252238] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252239] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252240] Node 0 DMA32 per-cpu:
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252242] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252243] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252246] active_anon:931188 inactive_anon:1396 isolated_anon:0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252247] active_file:0 inactive_file:3 isolated_file:0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252247] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252248] free:4048 slab_reclaimable:2588 slab_unreclaimable:3185
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252249] mapped:1362 shmem:1408 pagetables:4077 bounce:0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252250] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252257] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252260] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3724752kB inactive_anon:5584kB active_file:0kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5448kB shmem:5632kB slab_reclaimable:10352kB slab_unreclaimable:12740kB kernel_stack:944kB pagetables:16308kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:681 all_unreclaimable? yes
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252266] lowmem_reserve[]: 0 0 0 0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252269] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252276] Node 0 DMA32: 1417*4kB 16*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7908kB
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252282] 1424 total pagecache pages
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252283] 0 pages in swap cache
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252285] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252286] Free swap = 0kB
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.252287] Total swap = 0kB
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.257999] 985072 pages RAM
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258001] 24998 pages reserved
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258002] 4233 pages shared
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258004] 952865 pages non-shared
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258005] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258014] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258017] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258020] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258023] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258026] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258030] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258033] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258036] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258040] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258043] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258046] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258049] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258051] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258054] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258057] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258060] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258063] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258066] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258069] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258072] [ 3502] 1012 3502 77815 13677 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258075] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258077] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258080] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258083] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258086] [ 6563] 0 6563 89584 2503 1 0 0 php5-fpm
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258089] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258092] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258096] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258098] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258101] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258105] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258108] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258111] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258114] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258117] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258119] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258122] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258126] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258129] [ 2143] 108 2143 6795 75 0 0 0 trivial-rewrite
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258132] [ 2144] 108 2144 11107 159 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258135] [ 2148] 108 2148 11107 158 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258138] [ 2152] 108 2152 6792 69 1 0 0 scache
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258141] [ 2156] 108 2156 11108 159 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258144] [ 2160] 108 2160 11108 159 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258147] [ 2164] 108 2164 11108 159 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258150] [ 2168] 108 2168 11108 159 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258153] [ 2178] 108 2178 11108 161 1 0 0 smtp
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258156] [ 2585] 1012 2585 757325 694480 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258159] [ 3319] 1012 3319 225877 146992 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258161] [ 3324] 1012 3324 71045 8454 1 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258164] [ 3332] 1012 3332 71045 8455 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258167] [ 3399] 1012 3399 71043 8454 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258170] [ 3403] 1012 3403 1099 25 0 0 0 sh
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258173] [ 3404] 1012 3404 68572 6022 0 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258175] [ 3422] 1012 3422 70981 8425 1 0 0 php
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258178] [ 3466] 1012 3466 1099 26 0 0 0 sh
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258181] [ 3470] 1012 3470 5184 87 1 0 0 git
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258184] Out of memory: Kill process 2585 (php) score 724 or sacrifice child
Jul 16 13:19:48 ip-10-61-174-46 kernel: [1507684.258195] Killed process 2585 (php) total-vm:3029300kB, anon-rss:2777920kB, file-rss:0kB
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694746] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694751] php cpuset=/ mems_allowed=0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694755] Pid: 3504, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694756] Call Trace:
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694765] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694768] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694771] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694776] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694780] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694785] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694790] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694793] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694797] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694800] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694804] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694807] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694810] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694813] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694818] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694822] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694825] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694829] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694832] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694835] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694839] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694840] Mem-Info:
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694842] Node 0 DMA per-cpu:
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694844] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694845] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694846] Node 0 DMA32 per-cpu:
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694848] CPU 0: hi: 186, btch: 31 usd: 117
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694849] CPU 1: hi: 186, btch: 31 usd: 88
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694852] active_anon:931519 inactive_anon:1396 isolated_anon:0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694853] active_file:28 inactive_file:13 isolated_file:17
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694853] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694854] free:4033 slab_reclaimable:2472 slab_unreclaimable:3116
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694855] mapped:1369 shmem:1408 pagetables:3760 bounce:0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694856] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694863] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694866] Node 0 DMA32 free:7796kB min:7848kB low:9808kB high:11772kB active_anon:3726076kB inactive_anon:5584kB active_file:112kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):68kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5632kB slab_reclaimable:9888kB slab_unreclaimable:12464kB kernel_stack:904kB pagetables:15040kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:320 all_unreclaimable? yes
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694872] lowmem_reserve[]: 0 0 0 0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694875] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694882] Node 0 DMA32: 981*4kB 213*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7724kB
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694888] 1462 total pagecache pages
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694890] 0 pages in swap cache
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694891] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694892] Free swap = 0kB
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.694893] Total swap = 0kB
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700124] 985072 pages RAM
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700126] 24998 pages reserved
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700127] 4227 pages shared
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700129] 952718 pages non-shared
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700131] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700140] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700143] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700147] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700150] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700153] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700157] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700160] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700163] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700167] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700170] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700174] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700177] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700180] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700183] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700186] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700189] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700192] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700196] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700199] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700202] [ 3502] 1012 3502 77815 13684 0 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700205] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700208] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700211] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700214] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700217] [ 6563] 0 6563 89584 2511 0 0 0 php5-fpm
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700221] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700224] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700227] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700230] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700233] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700236] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700239] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700242] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700245] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700248] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700251] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700255] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700258] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700261] [ 2143] 108 2143 6795 75 1 0 0 trivial-rewrite
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700265] [ 2144] 108 2144 11107 159 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700268] [ 2148] 108 2148 11107 158 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700271] [ 2152] 108 2152 6792 69 1 0 0 scache
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700274] [ 2156] 108 2156 11108 159 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700277] [ 2160] 108 2160 11108 159 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700280] [ 2164] 108 2164 11108 159 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700283] [ 2168] 108 2168 11108 159 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700286] [ 2178] 108 2178 11108 161 1 0 0 smtp
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700289] [ 3319] 1012 3319 691270 628391 1 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700292] [ 3676] 1012 3676 71045 8475 0 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700295] [ 3680] 1012 3680 301681 238767 1 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700298] [ 3758] 1012 3758 68705 6100 0 0 0 php
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700301] Out of memory: Kill process 3319 (php) score 655 or sacrifice child
Jul 16 13:20:04 ip-10-61-174-46 kernel: [1507699.700314] Killed process 3319 (php) total-vm:2765080kB, anon-rss:2513564kB, file-rss:0kB
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108139] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108143] php cpuset=/ mems_allowed=0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108147] Pid: 3969, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108149] Call Trace:
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108156] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108174] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108181] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108184] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108188] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108191] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108194] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108197] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108199] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108202] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108204] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108208] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108210] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108213] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108215] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108218] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108220] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108222] Mem-Info:
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108223] Node 0 DMA per-cpu:
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108225] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108227] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108228] Node 0 DMA32 per-cpu:
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108229] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108231] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108233] active_anon:931350 inactive_anon:1396 isolated_anon:0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108234] active_file:0 inactive_file:9 isolated_file:0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108235] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108235] free:4048 slab_reclaimable:2511 slab_unreclaimable:3158
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108236] mapped:1361 shmem:1408 pagetables:4042 bounce:0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108237] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108244] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108247] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3725400kB inactive_anon:5584kB active_file:0kB inactive_file:36kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5444kB shmem:5632kB slab_reclaimable:10044kB slab_unreclaimable:12632kB kernel_stack:936kB pagetables:16168kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:193 all_unreclaimable? yes
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108253] lowmem_reserve[]: 0 0 0 0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108256] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108263] Node 0 DMA32: 821*4kB 323*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108269] 1448 total pagecache pages
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108270] 0 pages in swap cache
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108272] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108273] Free swap = 0kB
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.108274] Total swap = 0kB
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113504] 985072 pages RAM
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113506] 24998 pages reserved
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113508] 4228 pages shared
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113509] 952865 pages non-shared
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113511] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113519] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113523] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113526] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113529] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113532] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113535] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113539] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113542] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113546] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113549] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113552] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113556] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113559] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113562] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113565] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113568] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113571] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113574] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113577] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113580] [ 3502] 1012 3502 77815 13691 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113583] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113586] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113589] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113592] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113595] [ 6563] 0 6563 89584 2495 0 0 0 php5-fpm
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113598] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113602] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113605] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113607] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113610] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113614] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113617] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113620] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113623] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113626] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113629] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113632] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113635] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113639] [ 2143] 108 2143 6795 75 0 0 0 trivial-rewrite
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113642] [ 2144] 108 2144 11107 159 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113645] [ 2148] 108 2148 11107 158 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113648] [ 2152] 108 2152 6792 69 1 0 0 scache
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113651] [ 2156] 108 2156 11108 159 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113654] [ 2160] 108 2160 11108 159 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113657] [ 2164] 108 2164 11108 159 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113660] [ 2168] 108 2168 11108 159 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113663] [ 2178] 108 2178 11108 161 1 0 0 smtp
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113666] [ 3680] 1012 3680 771022 708155 0 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113669] [ 3969] 1012 3969 225749 145211 0 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113672] [ 4014] 1012 4014 71045 8453 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113675] [ 4042] 1012 4042 71045 8453 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113678] [ 4056] 1012 4056 69798 7191 0 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113681] [ 4066] 1012 4066 1099 26 0 0 0 sh
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113684] [ 4067] 1012 4067 65627 3614 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113686] [ 4075] 1012 4075 31064 384 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113690] [ 4078] 1012 4078 11720 42 1 0 0 php
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113692] Out of memory: Kill process 3680 (php) score 739 or sacrifice child
Jul 16 13:20:20 ip-10-61-174-46 kernel: [1507716.113704] Killed process 3680 (php) total-vm:3084088kB, anon-rss:2832620kB, file-rss:0kB
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236167] git invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236171] git cpuset=/ mems_allowed=0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236173] Pid: 4362, comm: git Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236175] Call Trace:
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236182] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236184] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236186] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236189] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236192] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236196] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236198] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236201] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236205] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236207] [<ffffffff81198d7f>] ? mntput+0x1f/0x30
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236210] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236213] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236215] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236218] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236221] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236224] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236227] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236230] [<ffffffff81179a4d>] ? vfs_read+0x11d/0x180
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236232] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236234] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236236] Mem-Info:
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236237] Node 0 DMA per-cpu:
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236239] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236240] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236241] Node 0 DMA32 per-cpu:
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236243] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236244] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236247] active_anon:929738 inactive_anon:1396 isolated_anon:0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236248] active_file:12 inactive_file:18 isolated_file:0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236248] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236249] free:3531 slab_reclaimable:2461 slab_unreclaimable:3594
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236250] mapped:1358 shmem:1408 pagetables:4202 bounce:0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236251] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236258] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236261] Node 0 DMA32 free:5788kB min:7848kB low:9808kB high:11772kB active_anon:3718952kB inactive_anon:5584kB active_file:48kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5632kB slab_reclaimable:9844kB slab_unreclaimable:14376kB kernel_stack:944kB pagetables:16808kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:412 all_unreclaimable? yes
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236267] lowmem_reserve[]: 0 0 0 0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236270] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236276] Node 0 DMA32: 561*4kB 162*8kB 13*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 5764kB
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236283] 1457 total pagecache pages
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236284] 0 pages in swap cache
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236285] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236287] Free swap = 0kB
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.236287] Total swap = 0kB
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241657] 985072 pages RAM
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241659] 24998 pages reserved
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241661] 4242 pages shared
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241662] 953391 pages non-shared
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241663] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241672] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241676] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241679] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241682] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241686] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241689] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241692] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241695] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241700] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241703] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241706] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241709] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241712] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241715] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241718] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241721] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241725] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241728] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241731] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241735] [ 3502] 1012 3502 77815 13697 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241738] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241741] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241744] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241747] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241750] [ 6563] 0 6563 89584 2532 1 0 0 php5-fpm
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241753] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241756] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241759] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241762] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241765] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241769] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241772] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241775] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241778] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241781] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241784] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241787] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241791] [ 2142] 108 2142 6818 74 1 0 0 cleanup
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241794] [ 2143] 108 2143 6795 75 0 0 0 trivial-rewrite
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241797] [ 2152] 108 2152 6792 69 0 0 0 scache
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241800] [ 2160] 108 2160 11108 158 0 0 0 smtp
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241803] [ 2178] 108 2178 11108 161 0 0 0 smtp
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241806] [ 3969] 1012 3969 636099 573188 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241810] [ 4056] 1012 4056 71043 8524 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241813] [ 4228] 1012 4228 268144 205244 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241816] [ 4229] 1012 4229 71045 8452 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241819] [ 4252] 1012 4252 129353 59849 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241822] [ 4280] 1012 4280 70981 8447 1 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241825] [ 4331] 1012 4331 1099 24 0 0 0 sh
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241828] [ 4332] 1012 4332 68424 5871 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241831] [ 4333] 1012 4333 1099 25 0 0 0 sh
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241834] [ 4335] 1012 4335 68424 5866 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241837] [ 4347] 1012 4347 1099 26 1 0 0 sh
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241840] [ 4348] 1012 4348 14084 62 0 0 0 git
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241843] [ 4349] 1012 4349 65832 3418 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241846] [ 4354] 1012 4354 63812 1551 0 0 0 php
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241849] [ 4361] 1012 4361 1099 25 0 0 0 sh
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241852] [ 4362] 1012 4362 4647 45 0 0 0 git
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241854] Out of memory: Kill process 3969 (php) score 598 or sacrifice child
Jul 16 13:20:34 ip-10-61-174-46 kernel: [1507730.241866] Killed process 3969 (php) total-vm:2544396kB, anon-rss:2292748kB, file-rss:4kB
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584168] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584173] php5-fpm cpuset=/ mems_allowed=0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584176] Pid: 6563, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584179] Call Trace:
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584187] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584190] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584193] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584197] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584203] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584207] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584211] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584214] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584218] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584222] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584226] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584229] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584233] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584236] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584240] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584244] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584249] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584253] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584256] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584258] Mem-Info:
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584260] Node 0 DMA per-cpu:
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584262] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584264] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584266] Node 0 DMA32 per-cpu:
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584268] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584270] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584273] active_anon:930749 inactive_anon:1396 isolated_anon:0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584274] active_file:29 inactive_file:31 isolated_file:0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584275] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584276] free:4047 slab_reclaimable:2616 slab_unreclaimable:3250
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584277] mapped:1354 shmem:1408 pagetables:4321 bounce:0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584279] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584287] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584291] Node 0 DMA32 free:7852kB min:7848kB low:9808kB high:11772kB active_anon:3722996kB inactive_anon:5584kB active_file:116kB inactive_file:124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5632kB slab_reclaimable:10464kB slab_unreclaimable:13000kB kernel_stack:968kB pagetables:17284kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:923 all_unreclaimable? yes
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584300] lowmem_reserve[]: 0 0 0 0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584303] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584312] Node 0 DMA32: 799*4kB 335*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8020kB
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584322] 1468 total pagecache pages
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584323] 0 pages in swap cache
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584325] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584326] Free swap = 0kB
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.584328] Total swap = 0kB
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591431] 985072 pages RAM
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591434] 24998 pages reserved
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591435] 4243 pages shared
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591436] 952824 pages non-shared
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591437] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591446] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591449] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591452] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591455] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591459] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591462] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591465] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591468] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591472] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591475] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591478] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591482] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591485] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591488] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591491] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591494] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591497] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591500] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591502] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591505] [ 3502] 1012 3502 77815 13698 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591508] [ 3504] 1012 3504 68832 6237 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591511] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591514] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591517] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591519] [ 6563] 0 6563 89584 2510 0 0 0 php5-fpm
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591522] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591525] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591528] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591530] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591533] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591537] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591540] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591543] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591546] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591549] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591552] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591555] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591559] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591562] [ 4056] 1012 4056 195921 119850 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591565] [ 4228] 1012 4228 501565 438706 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591568] [ 4252] 1012 4252 335793 272933 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591571] [ 4280] 1012 4280 71045 8504 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591574] [ 4559] 1012 4559 1099 24 1 0 0 sh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591577] [ 4560] 1012 4560 68424 5868 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591580] [ 4561] 1012 4561 1099 25 1 0 0 sh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591583] [ 4562] 1012 4562 68424 5871 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591586] [ 4569] 1012 4569 70657 8124 1 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591589] [ 4571] 1012 4571 57909 7554 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591592] [ 4583] 1012 4583 1099 25 0 0 0 sh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591594] [ 4584] 1012 4584 5216 86 0 0 0 git
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591597] [ 4585] 1012 4585 1099 25 1 0 0 sh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591600] [ 4586] 1012 4586 4669 64 0 0 0 git
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591603] [ 4587] 1012 4587 68476 5523 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591607] [ 4588] 1012 4588 68476 5523 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591610] [ 4597] 1012 4597 1099 25 1 0 0 sh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591613] [ 4598] 1012 4598 10901 165 1 0 0 ssh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591615] [ 4599] 1012 4599 1099 25 0 0 0 sh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591618] [ 4600] 1012 4600 65061 3028 0 0 0 php
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591621] [ 4601] 1012 4601 10901 175 0 0 0 ssh
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591624] [ 4613] 1012 4613 9799 72 1 0 0 git
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591627] Out of memory: Kill process 4228 (php) score 457 or sacrifice child
Jul 16 13:20:47 ip-10-61-174-46 kernel: [1507742.591639] Killed process 4228 (php) total-vm:2006260kB, anon-rss:1754824kB, file-rss:0kB
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653327] php5-fpm invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653333] php5-fpm cpuset=/ mems_allowed=0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653336] Pid: 6563, comm: php5-fpm Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653338] Call Trace:
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653346] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653350] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653353] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653356] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653361] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653365] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653368] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653371] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653374] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653377] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653380] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653382] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653385] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653388] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653391] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653395] [<ffffffff811bdaac>] ? ep_poll+0x1cc/0x380
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653399] [<ffffffff81060790>] ? try_to_wake_up+0x200/0x200
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653402] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653404] Mem-Info:
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653405] Node 0 DMA per-cpu:
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653407] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653408] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653409] Node 0 DMA32 per-cpu:
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653411] CPU 0: hi: 186, btch: 31 usd: 70
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653413] CPU 1: hi: 186, btch: 31 usd: 38
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653415] active_anon:931833 inactive_anon:1396 isolated_anon:0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653416] active_file:7 inactive_file:8 isolated_file:19
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653417] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653417] free:4025 slab_reclaimable:2552 slab_unreclaimable:3123
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653418] mapped:1356 shmem:1408 pagetables:3501 bounce:0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653420] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653426] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653429] Node 0 DMA32 free:7764kB min:7848kB low:9808kB high:11772kB active_anon:3727332kB inactive_anon:5584kB active_file:28kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5632kB slab_reclaimable:10208kB slab_unreclaimable:12492kB kernel_stack:840kB pagetables:14004kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:190 all_unreclaimable? yes
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653436] lowmem_reserve[]: 0 0 0 0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653438] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653445] Node 0 DMA32: 1429*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7764kB
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653452] 1429 total pagecache pages
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653453] 0 pages in swap cache
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653454] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653455] Free swap = 0kB
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.653456] Total swap = 0kB
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658649] 985072 pages RAM
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658652] 24998 pages reserved
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658653] 4238 pages shared
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658654] 952806 pages non-shared
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658656] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658665] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658668] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658671] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658674] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658678] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658681] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658684] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658688] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658692] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658695] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658698] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658701] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658704] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658707] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658710] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658713] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658717] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658720] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658723] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658726] [ 3502] 1012 3502 77879 13705 1 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658729] [ 3504] 1012 3504 68832 6236 1 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658733] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658736] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658739] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658742] [ 6563] 0 6563 89584 2494 0 0 0 php5-fpm
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658745] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658748] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658751] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658754] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658757] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658760] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658763] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658766] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658769] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658772] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658775] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658778] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658782] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658785] [ 4056] 1012 4056 451256 388412 0 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658788] [ 4252] 1012 4252 540798 477978 0 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658791] [ 4819] 1012 4819 71045 8456 1 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658794] [ 4840] 1012 4840 71045 8455 1 0 0 php
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658797] Out of memory: Kill process 4252 (php) score 498 or sacrifice child
Jul 16 13:20:59 ip-10-61-174-46 kernel: [1507754.658808] Killed process 4252 (php) total-vm:2163192kB, anon-rss:1911904kB, file-rss:8kB
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680152] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680157] php cpuset=/ mems_allowed=0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680161] Pid: 4840, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680163] Call Trace:
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680171] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680174] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680176] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680179] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680183] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680187] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680189] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680192] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680196] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680199] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680201] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680205] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680207] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680209] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680212] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680214] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680217] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680219] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680222] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680225] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680227] Mem-Info:
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680228] Node 0 DMA per-cpu:
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680230] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680231] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680232] Node 0 DMA32 per-cpu:
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680235] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680238] active_anon:931735 inactive_anon:1396 isolated_anon:0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680239] active_file:14 inactive_file:36 isolated_file:0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680239] unevictable:0 dirty:5 writeback:0 unstable:0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680240] free:4048 slab_reclaimable:2576 slab_unreclaimable:3115
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680241] mapped:1363 shmem:1408 pagetables:3678 bounce:0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680249] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680252] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3726940kB inactive_anon:5584kB active_file:56kB inactive_file:144kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:20kB writeback:0kB mapped:5452kB shmem:5632kB slab_reclaimable:10304kB slab_unreclaimable:12460kB kernel_stack:856kB pagetables:14712kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15813 all_unreclaimable? yes
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680258] lowmem_reserve[]: 0 0 0 0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680261] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680268] Node 0 DMA32: 796*4kB 337*8kB 2*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7960kB
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680274] 1457 total pagecache pages
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680275] 0 pages in swap cache
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680277] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680278] Free swap = 0kB
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.680279] Total swap = 0kB
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685870] 985072 pages RAM
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685872] 24998 pages reserved
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685873] 4248 pages shared
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685875] 952847 pages non-shared
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685877] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685885] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685889] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685892] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685895] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685898] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685901] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685904] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685908] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685912] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685915] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685918] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685921] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685924] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685927] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685931] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685934] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685937] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685940] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685943] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685947] [ 3502] 1012 3502 77879 13708 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685950] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685953] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685956] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685959] [ 3533] 1012 3533 68271 5638 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685962] [ 6563] 0 6563 89584 2526 0 0 0 php5-fpm
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685965] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685968] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685971] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685974] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685978] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685981] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685984] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685987] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685990] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685993] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685996] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.685999] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686003] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686006] [ 4056] 1012 4056 711368 648464 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686009] [ 4840] 1012 4840 274158 211270 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686012] [ 5025] 1012 5025 71045 8454 1 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686015] [ 5029] 1012 5029 71045 8455 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686018] [ 5057] 1012 5057 1099 24 0 0 0 sh
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686021] [ 5058] 1012 5058 68708 6091 0 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686024] [ 5139] 1012 5139 22609 345 1 0 0 php
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686027] [ 5140] 1012 5140 1099 26 0 0 0 sh
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686030] [ 5141] 1012 5141 19592 72 0 0 0 git
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686033] Out of memory: Kill process 4056 (php) score 676 or sacrifice child
Jul 16 13:21:10 ip-10-61-174-46 kernel: [1507765.686044] Killed process 4056 (php) total-vm:2845472kB, anon-rss:2593856kB, file-rss:0kB
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823055] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823060] php cpuset=/ mems_allowed=0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823064] Pid: 3504, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823066] Call Trace:
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823074] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823077] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823080] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823084] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823088] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823093] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823096] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823099] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823103] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823106] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823108] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823111] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823113] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823117] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823121] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823124] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823127] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823130] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823132] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823135] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823136] Mem-Info:
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823138] Node 0 DMA per-cpu:
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823140] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823141] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823142] Node 0 DMA32 per-cpu:
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823144] CPU 0: hi: 186, btch: 31 usd: 112
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823145] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823148] active_anon:932018 inactive_anon:1396 isolated_anon:0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823149] active_file:11 inactive_file:0 isolated_file:39
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823150] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823150] free:4043 slab_reclaimable:2469 slab_unreclaimable:3078
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823151] mapped:1369 shmem:1408 pagetables:3519 bounce:0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823152] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823159] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823162] Node 0 DMA32 free:7836kB min:7848kB low:9808kB high:11772kB active_anon:3728072kB inactive_anon:5584kB active_file:44kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):156kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5476kB shmem:5632kB slab_reclaimable:9876kB slab_unreclaimable:12312kB kernel_stack:840kB pagetables:14076kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:211 all_unreclaimable? yes
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823169] lowmem_reserve[]: 0 0 0 0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823171] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823178] Node 0 DMA32: 1367*4kB 30*8kB 3*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823185] 1451 total pagecache pages
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823186] 0 pages in swap cache
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823187] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823188] Free swap = 0kB
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.823189] Total swap = 0kB
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830339] 985072 pages RAM
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830341] 24998 pages reserved
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830343] 4234 pages shared
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830344] 952786 pages non-shared
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830345] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830354] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830357] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830360] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830363] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830366] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830369] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830372] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830376] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830379] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830382] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830385] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830388] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830391] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830394] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830397] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830399] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830402] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830405] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830408] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830411] [ 3502] 1012 3502 77879 13714 1 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830413] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830415] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830418] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830421] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830424] [ 6563] 0 6563 89584 2510 1 0 0 php5-fpm
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830427] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830429] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830432] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830435] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830437] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830440] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830443] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830446] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830449] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830452] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830455] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830459] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830462] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830465] [ 4840] 1012 4840 738569 675764 0 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830469] [ 5181] 1012 5181 253684 190823 0 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830472] [ 5291] 1012 5291 71045 8454 0 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830474] [ 5303] 1012 5303 71045 8452 1 0 0 php
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830477] Out of memory: Kill process 4840 (php) score 705 or sacrifice child
Jul 16 13:21:24 ip-10-61-174-46 kernel: [1507779.830488] Killed process 4840 (php) total-vm:2954276kB, anon-rss:2703048kB, file-rss:8kB
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240140] php cpuset=/ mems_allowed=0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240143] Pid: 5291, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240145] Call Trace:
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240159] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240162] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240166] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240171] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240178] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240183] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240186] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240190] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240193] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240197] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240200] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240204] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240208] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240212] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240214] Mem-Info:
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240216] Node 0 DMA per-cpu:
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240218] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240220] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240222] Node 0 DMA32 per-cpu:
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240224] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240226] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240229] active_anon:931495 inactive_anon:1396 isolated_anon:0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240230] active_file:28 inactive_file:32 isolated_file:0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240231] unevictable:0 dirty:7 writeback:0 unstable:0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240232] free:2955 slab_reclaimable:2522 slab_unreclaimable:3400
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240233] mapped:1371 shmem:1408 pagetables:3529 bounce:0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240244] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240248] Node 0 DMA32 free:3484kB min:7848kB low:9808kB high:11772kB active_anon:3725980kB inactive_anon:5584kB active_file:112kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:28kB writeback:0kB mapped:5484kB shmem:5632kB slab_reclaimable:10088kB slab_unreclaimable:13600kB kernel_stack:824kB pagetables:14116kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1399 all_unreclaimable? yes
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240256] lowmem_reserve[]: 0 0 0 0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240260] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240269] Node 0 DMA32: 215*4kB 118*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3484kB
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240279] 1467 total pagecache pages
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240280] 0 pages in swap cache
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240282] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240284] Free swap = 0kB
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.240285] Total swap = 0kB
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247539] 985072 pages RAM
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247541] 24998 pages reserved
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247543] 4231 pages shared
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247544] 953988 pages non-shared
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247546] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247555] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247559] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247562] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247565] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247568] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247571] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247575] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247578] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247582] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247585] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247588] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247591] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247594] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247597] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247600] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247603] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247606] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247610] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247613] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247616] [ 3502] 1012 3502 77879 13721 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247619] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247622] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247625] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247628] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247631] [ 6563] 0 6563 89584 2493 0 0 0 php5-fpm
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247635] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247638] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247641] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247644] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247647] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247650] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247653] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247656] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247659] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247662] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247665] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247668] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247672] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247675] [ 5181] 1012 5181 620546 557658 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247678] [ 5291] 1012 5291 71109 8568 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247681] [ 5458] 1012 5458 327856 264972 0 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247684] [ 5751] 1012 5751 120800 51663 1 0 0 php
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247687] Out of memory: Kill process 5181 (php) score 582 or sacrifice child
Jul 16 13:21:40 ip-10-61-174-46 kernel: [1507796.247700] Killed process 5181 (php) total-vm:2482184kB, anon-rss:2230628kB, file-rss:4kB
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330080] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330085] php cpuset=/ mems_allowed=0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330088] Pid: 5885, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330090] Call Trace:
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330098] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330101] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330104] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330108] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330112] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330116] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330119] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330123] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330126] [<ffffffff8113ca82>] ? do_wp_page+0x382/0x740
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330129] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330133] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330136] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330140] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330143] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330146] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330150] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330153] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330156] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330159] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330162] Mem-Info:
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330163] Node 0 DMA per-cpu:
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330165] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330167] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330168] Node 0 DMA32 per-cpu:
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330171] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330172] CPU 1: hi: 186, btch: 31 usd: 10
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330175] active_anon:932106 inactive_anon:1396 isolated_anon:0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330176] active_file:12 inactive_file:19 isolated_file:0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330177] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330178] free:4048 slab_reclaimable:2442 slab_unreclaimable:3035
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330179] mapped:1357 shmem:1408 pagetables:3530 bounce:0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330181] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330190] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330194] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3728424kB inactive_anon:5584kB active_file:48kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5428kB shmem:5632kB slab_reclaimable:9768kB slab_unreclaimable:12140kB kernel_stack:824kB pagetables:14120kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:191 all_unreclaimable? yes
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330202] lowmem_reserve[]: 0 0 0 0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330205] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330215] Node 0 DMA32: 1109*4kB 167*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7932kB
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330224] 1481 total pagecache pages
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330226] 0 pages in swap cache
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330228] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330229] Free swap = 0kB
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.330230] Total swap = 0kB
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335612] 985072 pages RAM
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335615] 24998 pages reserved
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335616] 4240 pages shared
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335617] 952846 pages non-shared
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335619] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335628] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335631] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335635] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335638] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335641] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335644] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335648] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335651] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335655] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335658] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335661] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335665] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335668] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335671] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335674] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335677] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335680] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335683] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335687] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335690] [ 3502] 1012 3502 77879 13724 1 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335693] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335696] [ 3506] 1012 3506 69239 6606 0 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335699] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335702] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335705] [ 6563] 0 6563 89584 2531 0 0 0 php5-fpm
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335709] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335712] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335715] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335718] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335721] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335724] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335727] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335730] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335734] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335737] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335740] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335743] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335747] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335750] [ 5458] 1012 5458 564286 501422 0 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335753] [ 5751] 1012 5751 428025 365194 0 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335756] [ 5843] 1012 5843 71045 8478 0 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335759] [ 5885] 1012 5885 71045 8452 1 0 0 php
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335762] Out of memory: Kill process 5458 (php) score 523 or sacrifice child
Jul 16 13:21:52 ip-10-61-174-46 kernel: [1507808.335773] Killed process 5458 (php) total-vm:2257144kB, anon-rss:2005688kB, file-rss:0kB
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608147] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608152] php cpuset=/ mems_allowed=0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608155] Pid: 5977, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608157] Call Trace:
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608165] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608172] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608176] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608181] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608184] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608189] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608192] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608196] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608199] [<ffffffff8113ca82>] ? do_wp_page+0x382/0x740
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608201] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608204] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608206] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608209] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608211] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608213] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608216] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608219] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608223] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608226] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608228] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608230] Mem-Info:
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608231] Node 0 DMA per-cpu:
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608233] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608235] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608236] Node 0 DMA32 per-cpu:
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608237] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608239] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608241] active_anon:929745 inactive_anon:1396 isolated_anon:0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608242] active_file:25 inactive_file:30 isolated_file:0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608242] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608243] free:3923 slab_reclaimable:2496 slab_unreclaimable:3522
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608244] mapped:1395 shmem:1408 pagetables:3923 bounce:0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608245] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608252] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608255] Node 0 DMA32 free:7356kB min:7848kB low:9808kB high:11772kB active_anon:3718980kB inactive_anon:5584kB active_file:100kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5580kB shmem:5632kB slab_reclaimable:9984kB slab_unreclaimable:14088kB kernel_stack:880kB pagetables:15692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1308 all_unreclaimable? yes
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608261] lowmem_reserve[]: 0 0 0 0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608264] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608271] Node 0 DMA32: 1185*4kB 62*8kB 17*16kB 0*32kB 1*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 7364kB
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608277] 1458 total pagecache pages
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608279] 0 pages in swap cache
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608280] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608281] Free swap = 0kB
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.608282] Total swap = 0kB
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614009] 985072 pages RAM
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614012] 24998 pages reserved
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614013] 4240 pages shared
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614014] 953002 pages non-shared
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614016] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614024] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614028] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614031] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614034] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614037] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614040] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614044] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614047] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614051] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614054] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614057] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614061] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614064] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614067] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614070] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614073] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614076] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614080] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614083] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614086] [ 3502] 1012 3502 77879 13727 1 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614089] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614092] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614095] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614098] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614101] [ 6563] 0 6563 89584 2515 0 0 0 php5-fpm
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614104] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614108] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614111] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614114] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614117] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614120] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614123] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614126] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614129] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614132] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614135] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614139] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614142] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614146] [ 5751] 1012 5751 616324 553476 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614149] [ 5843] 1012 5843 301294 238387 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614152] [ 5975] 1012 5975 120429 50799 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614155] [ 5977] 1012 5977 70788 8201 1 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614157] [ 6004] 1012 6004 72977 10411 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614160] [ 6013] 1012 6013 71045 8453 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614163] [ 6060] 1012 6060 1099 24 0 0 0 sh
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614166] [ 6061] 1012 6061 68424 5870 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614169] [ 6085] 1012 6085 1099 25 0 0 0 sh
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614172] [ 6086] 1012 6086 14084 62 0 0 0 git
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614175] [ 6087] 1012 6087 68400 5438 0 0 0 php
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614178] Out of memory: Kill process 5751 (php) score 577 or sacrifice child
Jul 16 13:22:02 ip-10-61-174-46 kernel: [1507817.614189] Killed process 5751 (php) total-vm:2465296kB, anon-rss:2213904kB, file-rss:0kB
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805940] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805945] php cpuset=/ mems_allowed=0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805948] Pid: 3504, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805950] Call Trace:
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805958] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805961] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805964] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805968] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805972] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805977] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805980] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805984] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805987] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805991] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805995] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.805998] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806002] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806006] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806010] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806014] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806017] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806021] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806025] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806028] Mem-Info:
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806029] Node 0 DMA per-cpu:
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806032] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806034] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806035] Node 0 DMA32 per-cpu:
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806038] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806040] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806043] active_anon:931175 inactive_anon:1396 isolated_anon:0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806044] active_file:18 inactive_file:1 isolated_file:0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806045] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806046] free:2822 slab_reclaimable:2535 slab_unreclaimable:3458
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806047] mapped:1356 shmem:1408 pagetables:3802 bounce:0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806049] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806058] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806062] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724700kB inactive_anon:5584kB active_file:72kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5632kB slab_reclaimable:10140kB slab_unreclaimable:13832kB kernel_stack:848kB pagetables:15208kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:114 all_unreclaimable? yes
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806070] lowmem_reserve[]: 0 0 0 0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806073] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806083] Node 0 DMA32: 273*4kB 1*8kB 0*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2956kB
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806093] 1427 total pagecache pages
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806094] 0 pages in swap cache
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806096] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806097] Free swap = 0kB
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.806099] Total swap = 0kB
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811775] 985072 pages RAM
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811777] 24998 pages reserved
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811779] 4227 pages shared
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811780] 954121 pages non-shared
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811782] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811791] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811794] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811798] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811801] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811804] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811807] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811810] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811814] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811818] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811821] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811824] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811827] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811830] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811833] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811837] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811840] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811843] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811846] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811849] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811853] [ 3502] 1012 3502 77879 13730 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811856] [ 3504] 1012 3504 68832 6236 0 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811859] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811862] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811865] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811868] [ 6563] 0 6563 89584 2499 1 0 0 php5-fpm
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811872] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811875] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811878] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811881] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811884] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811887] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811890] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811894] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811897] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811900] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811903] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811906] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811909] [ 2152] 108 2152 6792 78 0 0 0 scache
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811913] [ 5843] 1012 5843 532861 470006 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811916] [ 5975] 1012 5975 365875 303010 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811919] [ 6004] 1012 6004 73041 10489 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811922] [ 6120] 1012 6120 71045 8477 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811925] [ 6192] 1012 6192 140769 73779 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811928] [ 6197] 1012 6197 70981 8423 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811931] [ 6236] 1012 6236 70910 8363 1 0 0 php
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811934] Out of memory: Kill process 5843 (php) score 490 or sacrifice child
Jul 16 13:22:13 ip-10-61-174-46 kernel: [1507828.811944] Killed process 5843 (php) total-vm:2131444kB, anon-rss:1880024kB, file-rss:0kB
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375071] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375075] php cpuset=/ mems_allowed=0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375079] Pid: 6365, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375081] Call Trace:
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375088] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375092] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375095] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375099] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375105] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375109] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375113] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375116] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375120] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375124] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375127] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375129] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375132] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375135] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375137] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375140] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375142] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375144] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375148] [<ffffffff81013728>] ? __switch_to+0x138/0x360
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375151] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375154] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375156] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375158] Mem-Info:
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375159] Node 0 DMA per-cpu:
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375161] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375163] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375164] Node 0 DMA32 per-cpu:
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375165] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375167] CPU 1: hi: 186, btch: 31 usd: 28
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375169] active_anon:931923 inactive_anon:1396 isolated_anon:0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375170] active_file:0 inactive_file:13 isolated_file:0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375171] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375171] free:4048 slab_reclaimable:2442 slab_unreclaimable:3050
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375172] mapped:1360 shmem:1408 pagetables:3696 bounce:0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375174] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375180] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375183] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3727692kB inactive_anon:5584kB active_file:0kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5632kB slab_reclaimable:9768kB slab_unreclaimable:12200kB kernel_stack:856kB pagetables:14784kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:127 all_unreclaimable? yes
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375189] lowmem_reserve[]: 0 0 0 0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375192] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375199] Node 0 DMA32: 532*4kB 455*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7928kB
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375205] 1424 total pagecache pages
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375207] 0 pages in swap cache
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375208] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375209] Free swap = 0kB
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.375210] Total swap = 0kB
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380462] 985072 pages RAM
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380465] 24998 pages reserved
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380466] 4230 pages shared
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380467] 952846 pages non-shared
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380469] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380478] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380481] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380484] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380487] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380491] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380494] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380497] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380501] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380505] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380508] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380511] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380514] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380517] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380520] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380524] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380527] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380530] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380533] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380536] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380539] [ 3502] 1012 3502 77879 13736 1 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380542] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380546] [ 3506] 1012 3506 69239 6606 0 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380549] [ 3526] 1012 3526 69673 5532 0 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380552] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380555] [ 6563] 0 6563 89584 2535 0 0 0 php5-fpm
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380558] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380561] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380564] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380567] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380570] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380574] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380577] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380580] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380583] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380586] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380589] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380593] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380596] [ 5975] 1012 5975 592705 529836 0 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380599] [ 6192] 1012 6192 388980 326095 1 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380602] [ 6365] 1012 6365 71045 8454 0 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380605] [ 6421] 1012 6421 70910 8364 1 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380608] [ 6443] 1012 6443 1099 25 0 0 0 sh
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380611] [ 6444] 1012 6444 68424 5873 0 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380614] [ 6456] 1012 6456 1099 25 1 0 0 sh
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380617] [ 6457] 1012 6457 4682 58 1 0 0 git
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380620] [ 6458] 1012 6458 67693 4704 0 0 0 php
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380623] Out of memory: Kill process 5975 (php) score 553 or sacrifice child
Jul 16 13:22:24 ip-10-61-174-46 kernel: [1507839.380634] Killed process 5975 (php) total-vm:2370820kB, anon-rss:2119344kB, file-rss:0kB
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816279] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816284] php cpuset=/ mems_allowed=0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816287] Pid: 6192, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816289] Call Trace:
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816297] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816300] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816303] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816307] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816311] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816315] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816318] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816322] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816326] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816329] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816333] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816336] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816339] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816342] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816345] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816348] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816352] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816355] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816358] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816361] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816363] Mem-Info:
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816364] Node 0 DMA per-cpu:
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816366] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816367] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816368] Node 0 DMA32 per-cpu:
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816370] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816371] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816374] active_anon:930624 inactive_anon:1396 isolated_anon:0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816374] active_file:10 inactive_file:8 isolated_file:0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816375] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816376] free:3523 slab_reclaimable:2543 slab_unreclaimable:3446
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816376] mapped:1356 shmem:1408 pagetables:3609 bounce:0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816378] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816385] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816388] Node 0 DMA32 free:5756kB min:7848kB low:9808kB high:11772kB active_anon:3722496kB inactive_anon:5584kB active_file:40kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5632kB slab_reclaimable:10172kB slab_unreclaimable:13784kB kernel_stack:824kB pagetables:14436kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:123 all_unreclaimable? yes
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816394] lowmem_reserve[]: 0 0 0 0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816397] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816403] Node 0 DMA32: 888*4kB 114*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 5760kB
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816410] 1426 total pagecache pages
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816411] 0 pages in swap cache
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816412] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816413] Free swap = 0kB
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.816414] Total swap = 0kB
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821700] 985072 pages RAM
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821703] 24998 pages reserved
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821704] 4226 pages shared
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821705] 953424 pages non-shared
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821707] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821716] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821719] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821722] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821725] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821729] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821732] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821735] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821739] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821742] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821746] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821749] [ 831] 0 831 4226 41 1 0 0 atd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821752] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821755] [ 983] 108 983 6834 107 0 0 0 qmgr
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821758] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821762] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821765] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821768] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821771] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821774] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821777] [ 3502] 1012 3502 77879 13744 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821780] [ 3504] 1012 3504 68832 6237 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821784] [ 3506] 1012 3506 69239 6607 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821787] [ 3526] 1012 3526 69673 5532 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821790] [ 3533] 1012 3533 68271 5638 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821793] [ 6563] 0 6563 89584 2519 1 0 0 php5-fpm
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821797] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821800] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821803] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821806] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821809] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821812] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821815] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821818] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821821] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821824] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821827] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821831] [12591] 108 12591 6792 77 0 0 0 pickup
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821834] [ 6192] 1012 6192 864593 801784 0 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821837] [ 6815] 1012 6815 71043 8458 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821840] [ 6838] 1012 6838 126759 57682 0 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821844] [ 6881] 1012 6881 70314 7747 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821847] [ 6884] 1012 6884 68962 6386 1 0 0 php
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821850] Out of memory: Kill process 6192 (php) score 836 or sacrifice child
Jul 16 13:22:40 ip-10-61-174-46 kernel: [1507855.821861] Killed process 6192 (php) total-vm:3458372kB, anon-rss:3207136kB, file-rss:0kB
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584467] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584471] php cpuset=/ mems_allowed=0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584473] Pid: 10438, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584475] Call Trace:
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584482] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584484] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584487] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584489] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584493] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584496] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584498] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584501] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584504] [<ffffffff8113c9e8>] ? do_wp_page+0x2e8/0x740
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584506] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584509] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584512] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584514] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584516] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584520] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584523] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584527] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584529] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584532] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584534] Mem-Info:
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584535] Node 0 DMA per-cpu:
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584537] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584538] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584539] Node 0 DMA32 per-cpu:
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584541] CPU 0: hi: 186, btch: 31 usd: 49
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584542] CPU 1: hi: 186, btch: 31 usd: 33
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584545] active_anon:930701 inactive_anon:1398 isolated_anon:0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584546] active_file:0 inactive_file:22 isolated_file:0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584546] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584547] free:4020 slab_reclaimable:2756 slab_unreclaimable:3128
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584548] mapped:1384 shmem:1409 pagetables:4218 bounce:0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584549] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584556] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584559] Node 0 DMA32 free:7744kB min:7848kB low:9808kB high:11772kB active_anon:3722804kB inactive_anon:5592kB active_file:0kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5536kB shmem:5636kB slab_reclaimable:11024kB slab_unreclaimable:12512kB kernel_stack:1064kB pagetables:16872kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:199 all_unreclaimable? yes
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584565] lowmem_reserve[]: 0 0 0 0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584568] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584575] Node 0 DMA32: 1054*4kB 185*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7744kB
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584581] 1477 total pagecache pages
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584583] 0 pages in swap cache
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584584] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584585] Free swap = 0kB
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.584586] Total swap = 0kB
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590079] 985072 pages RAM
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590082] 24998 pages reserved
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590083] 4411 pages shared
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590084] 952633 pages non-shared
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590086] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590095] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590098] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590101] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590104] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590108] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590111] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590114] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590118] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590122] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590125] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590128] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590131] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590134] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590137] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590140] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590143] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590146] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590150] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590153] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590156] [ 6563] 0 6563 89584 2490 1 0 0 php5-fpm
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590159] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590162] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590165] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590168] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590171] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590174] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590177] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590180] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590183] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590186] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590189] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590193] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590196] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590199] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590202] [ 8228] 1012 8228 70351 5769 0 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590205] [ 8230] 1012 8230 69183 6210 0 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590208] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590211] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590214] [ 8253] 1012 8253 68618 5624 1 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590217] [ 9132] 108 9132 6795 70 0 0 0 trivial-rewrite
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590220] [ 9133] 108 9133 11107 161 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590223] [ 9137] 108 9137 11107 159 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590226] [ 9183] 108 9183 6825 94 1 0 0 scache
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590229] [ 9189] 108 9189 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590232] [ 9199] 108 9199 11108 161 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590235] [ 9211] 108 9211 11108 159 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590238] [ 9217] 108 9217 11108 161 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590241] [ 9225] 108 9225 11108 161 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590244] [ 9239] 108 9239 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590247] [ 9259] 108 9259 11108 161 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590251] [ 9366] 108 9366 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590254] [ 9775] 1012 9775 926899 863689 1 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590257] [ 9979] 108 9979 6818 73 0 0 0 cleanup
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590260] [ 9981] 108 9981 6792 70 0 0 0 pickup
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590263] [10019] 108 10019 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590266] [10037] 108 10037 11108 158 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590269] [10042] 108 10042 11108 161 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590272] [10051] 108 10051 11108 157 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590275] [10238] 108 10238 6818 74 0 0 0 cleanup
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590278] [10371] 108 10371 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590281] [10375] 108 10375 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590284] [10381] 108 10381 11108 159 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590287] [10391] 108 10391 11108 159 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590290] [10392] 108 10392 11108 160 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590293] [10393] 108 10393 11108 159 1 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590296] [10394] 108 10394 11108 159 0 0 0 smtp
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590299] [10395] 1012 10395 75912 12987 0 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590302] [10437] 1012 10437 1099 24 1 0 0 sh
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590305] [10438] 1012 10438 68825 5875 0 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590308] [10444] 1012 10444 1099 25 1 0 0 sh
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590311] [10445] 1012 10445 4724 62 0 0 0 git
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590314] [10446] 1012 10446 64212 1629 1 0 0 php
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590317] Out of memory: Kill process 9775 (php) score 901 or sacrifice child
Jul 16 17:00:25 ip-10-61-174-46 kernel: [1520920.590329] Killed process 9775 (php) total-vm:3707596kB, anon-rss:3454756kB, file-rss:0kB
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841535] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841540] php cpuset=/ mems_allowed=0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841543] Pid: 8252, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841545] Call Trace:
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841553] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841556] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841559] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841563] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841567] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841572] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841574] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841578] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841581] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841585] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841588] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841591] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841595] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841600] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841604] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841607] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841611] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841613] Mem-Info:
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841615] Node 0 DMA per-cpu:
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841618] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841619] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841621] Node 0 DMA32 per-cpu:
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841623] CPU 0: hi: 186, btch: 31 usd: 84
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841625] CPU 1: hi: 186, btch: 31 usd: 16
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841628] active_anon:931174 inactive_anon:1398 isolated_anon:0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841629] active_file:7 inactive_file:29 isolated_file:0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841630] unevictable:0 dirty:6 writeback:0 unstable:0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841630] free:4029 slab_reclaimable:2517 slab_unreclaimable:3114
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841632] mapped:1367 shmem:1409 pagetables:4132 bounce:0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841634] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841641] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841644] Node 0 DMA32 free:7780kB min:7848kB low:9808kB high:11772kB active_anon:3724696kB inactive_anon:5592kB active_file:28kB inactive_file:116kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:24kB writeback:0kB mapped:5468kB shmem:5636kB slab_reclaimable:10068kB slab_unreclaimable:12456kB kernel_stack:1056kB pagetables:16528kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:102 all_unreclaimable? yes
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841651] lowmem_reserve[]: 0 0 0 0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841653] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841660] Node 0 DMA32: 377*4kB 159*8kB 110*16kB 30*32kB 4*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7804kB
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841667] 1454 total pagecache pages
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841668] 0 pages in swap cache
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841669] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841671] Free swap = 0kB
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.841671] Total swap = 0kB
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847117] 985072 pages RAM
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847119] 24998 pages reserved
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847120] 4391 pages shared
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847121] 952625 pages non-shared
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847123] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847132] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847135] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847139] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847142] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847145] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847148] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847151] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847155] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847159] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847162] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847165] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847169] [ 975] 0 975 6276 80 1 0 0 master
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847172] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847175] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847178] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847181] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847184] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847188] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847191] [15657] 0 15657 4777 60 1 0 0 cron
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847194] [ 6563] 0 6563 89584 2529 1 0 0 php5-fpm
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847197] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847200] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847203] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847206] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847209] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847212] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847216] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847219] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847222] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847225] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847228] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847231] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847234] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847237] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847241] [ 8228] 1012 8228 70351 5770 1 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847244] [ 8230] 1012 8230 69183 6210 1 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847247] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847249] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847252] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847255] [ 9132] 108 9132 6795 70 0 0 0 trivial-rewrite
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847259] [ 9133] 108 9133 11107 161 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847262] [ 9137] 108 9137 11107 159 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847265] [ 9183] 108 9183 6825 94 0 0 0 scache
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847268] [ 9189] 108 9189 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847271] [ 9199] 108 9199 11108 161 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847274] [ 9211] 108 9211 11108 159 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847277] [ 9217] 108 9217 11108 161 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847280] [ 9225] 108 9225 11108 161 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847283] [ 9239] 108 9239 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847286] [ 9259] 108 9259 11108 161 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847289] [ 9366] 108 9366 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847293] [ 9979] 108 9979 6818 73 0 0 0 cleanup
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847295] [ 9981] 108 9981 6792 70 0 0 0 pickup
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847299] [10019] 108 10019 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847302] [10037] 108 10037 11108 158 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847305] [10042] 108 10042 11108 161 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847308] [10051] 108 10051 11108 157 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847311] [10238] 108 10238 6818 74 0 0 0 cleanup
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847314] [10371] 108 10371 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847317] [10375] 108 10375 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847320] [10381] 108 10381 11108 159 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847323] [10391] 108 10391 11108 159 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847326] [10392] 108 10392 11108 160 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847329] [10393] 108 10393 11108 159 1 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847332] [10394] 108 10394 11108 159 0 0 0 smtp
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847335] [10473] 1012 10473 706981 643751 0 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847338] [10600] 1012 10600 295821 232513 1 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847341] [10633] 1012 10633 71455 8465 1 0 0 php
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847344] Out of memory: Kill process 10473 (php) score 671 or sacrifice child
Jul 16 17:00:46 ip-10-61-174-46 kernel: [1520941.847356] Killed process 10473 (php) total-vm:2827924kB, anon-rss:2574988kB, file-rss:16kB
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740759] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740764] php cpuset=/ mems_allowed=0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740768] Pid: 8230, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740770] Call Trace:
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740779] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740782] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740785] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740789] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740793] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740797] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740800] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740804] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740807] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740811] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740815] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740818] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740821] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740826] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740831] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740835] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740838] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740841] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740844] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740848] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740850] Mem-Info:
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740851] Node 0 DMA per-cpu:
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740853] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740855] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740856] Node 0 DMA32 per-cpu:
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740859] CPU 0: hi: 186, btch: 31 usd: 84
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740860] CPU 1: hi: 186, btch: 31 usd: 108
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740864] active_anon:931123 inactive_anon:1398 isolated_anon:0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740865] active_file:15 inactive_file:13 isolated_file:42
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740866] unevictable:0 dirty:17 writeback:0 unstable:0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740867] free:4037 slab_reclaimable:2324 slab_unreclaimable:3120
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740868] mapped:1435 shmem:1409 pagetables:4181 bounce:0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740870] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740879] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740883] Node 0 DMA32 free:7812kB min:7848kB low:9808kB high:11772kB active_anon:3724492kB inactive_anon:5592kB active_file:60kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):168kB present:3862656kB mlocked:0kB dirty:68kB writeback:0kB mapped:5740kB shmem:5636kB slab_reclaimable:9296kB slab_unreclaimable:12480kB kernel_stack:1032kB pagetables:16724kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:384 all_unreclaimable? yes
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740892] lowmem_reserve[]: 0 0 0 0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740895] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740905] Node 0 DMA32: 87*4kB 119*8kB 131*16kB 26*32kB 4*64kB 2*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740915] 1475 total pagecache pages
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740917] 0 pages in swap cache
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740919] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740920] Free swap = 0kB
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.740921] Total swap = 0kB
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746513] 985072 pages RAM
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746515] 24998 pages reserved
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746516] 4410 pages shared
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746517] 952484 pages non-shared
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746519] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746528] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746531] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746534] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746538] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746541] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746544] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746547] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746551] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746555] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746559] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746562] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746565] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746568] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746571] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746574] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746577] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746580] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746584] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746587] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746590] [ 6563] 0 6563 89584 2497 0 0 0 php5-fpm
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746593] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746597] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746600] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746603] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746606] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746609] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746612] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746615] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746618] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746621] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746624] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746628] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746631] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746634] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746637] [ 8228] 1012 8228 70351 5777 1 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746640] [ 8230] 1012 8230 69183 6209 0 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746643] [ 8232] 1012 8232 69859 6866 0 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746646] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746649] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746652] [ 9132] 108 9132 6795 70 1 0 0 trivial-rewrite
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746655] [ 9133] 108 9133 11107 161 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746658] [ 9137] 108 9137 11107 159 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746661] [ 9183] 108 9183 6825 94 0 0 0 scache
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746664] [ 9189] 108 9189 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746667] [ 9199] 108 9199 11108 161 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746670] [ 9211] 108 9211 11108 159 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746673] [ 9217] 108 9217 11108 161 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746676] [ 9225] 108 9225 11108 161 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746679] [ 9239] 108 9239 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746682] [ 9259] 108 9259 11108 161 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746685] [ 9366] 108 9366 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746688] [ 9979] 108 9979 6818 73 0 0 0 cleanup
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746691] [ 9981] 108 9981 6792 70 0 0 0 pickup
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746694] [10019] 108 10019 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746697] [10037] 108 10037 11108 158 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746700] [10042] 108 10042 11108 161 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746703] [10051] 108 10051 11108 157 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746706] [10238] 108 10238 6818 74 0 0 0 cleanup
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746710] [10371] 108 10371 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746713] [10375] 108 10375 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746715] [10381] 108 10381 11108 159 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746718] [10391] 108 10391 11108 159 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746721] [10392] 108 10392 11108 160 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746724] [10393] 108 10393 11108 159 1 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746727] [10394] 108 10394 11108 159 0 0 0 smtp
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746730] [10600] 1012 10600 944116 880876 1 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746733] [11046] 1012 11046 64275 1685 1 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746736] [11047] 1012 11047 64275 1684 1 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746739] [11052] 1012 11052 31456 392 0 0 0 php
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746742] Out of memory: Kill process 10600 (php) score 919 or sacrifice child
Jul 16 17:01:05 ip-10-61-174-46 kernel: [1520960.746754] Killed process 10600 (php) total-vm:3776464kB, anon-rss:3523500kB, file-rss:4kB
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872405] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872409] php cpuset=/ mems_allowed=0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872412] Pid: 8230, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872414] Call Trace:
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872421] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872423] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872426] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872428] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872432] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872435] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872438] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872441] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872444] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872447] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872449] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872451] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872454] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872458] [<ffffffff8108f420>] ? lock_hrtimer_base.isra.25+0x30/0x60
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872461] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872464] [<ffffffff8165d304>] ? do_nanosleep+0xa4/0xd0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872467] [<ffffffff81090288>] ? hrtimer_nanosleep+0xb8/0x180
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872470] [<ffffffff8108ee80>] ? update_rmtp+0x70/0x70
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872472] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872475] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872477] Mem-Info:
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872478] Node 0 DMA per-cpu:
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872480] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872481] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872482] Node 0 DMA32 per-cpu:
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872484] CPU 0: hi: 186, btch: 31 usd: 56
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872486] CPU 1: hi: 186, btch: 31 usd: 17
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872488] active_anon:931321 inactive_anon:1398 isolated_anon:0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872489] active_file:28 inactive_file:8 isolated_file:15
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872490] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872490] free:4022 slab_reclaimable:2359 slab_unreclaimable:3157
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872491] mapped:1367 shmem:1409 pagetables:4053 bounce:0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872493] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872499] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872502] Node 0 DMA32 free:7752kB min:7848kB low:9808kB high:11772kB active_anon:3725284kB inactive_anon:5592kB active_file:112kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):60kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5468kB shmem:5636kB slab_reclaimable:9436kB slab_unreclaimable:12628kB kernel_stack:1032kB pagetables:16212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:302 all_unreclaimable? yes
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872509] lowmem_reserve[]: 0 0 0 0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872511] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872518] Node 0 DMA32: 456*4kB 155*8kB 49*16kB 20*32kB 5*64kB 1*128kB 3*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7752kB
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872525] 1447 total pagecache pages
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872526] 0 pages in swap cache
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872527] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872528] Free swap = 0kB
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.872529] Total swap = 0kB
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878043] 985072 pages RAM
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878046] 24998 pages reserved
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878047] 4378 pages shared
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878048] 952675 pages non-shared
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878050] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878059] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878063] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878066] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878069] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878072] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878075] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878079] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878082] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878086] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878089] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878092] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878096] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878099] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878102] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878105] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878108] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878111] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878114] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878118] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878121] [ 6563] 0 6563 89584 2522 0 0 0 php5-fpm
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878124] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878127] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878130] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878133] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878136] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878139] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878143] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878146] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878149] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878152] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878155] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878158] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878161] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878164] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878168] [ 8228] 1012 8228 70415 5795 1 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878171] [ 8230] 1012 8230 69183 6209 1 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878174] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878176] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878179] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878183] [ 9132] 108 9132 6795 75 1 0 0 trivial-rewrite
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878186] [ 9133] 108 9133 11107 161 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878189] [ 9137] 108 9137 11107 159 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878192] [ 9183] 108 9183 6825 94 0 0 0 scache
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878195] [ 9189] 108 9189 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878199] [ 9199] 108 9199 11108 161 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878202] [ 9211] 108 9211 11108 159 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878205] [ 9217] 108 9217 11108 161 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878208] [ 9225] 108 9225 11108 161 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878211] [ 9239] 108 9239 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878214] [ 9259] 108 9259 11108 161 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878217] [ 9366] 108 9366 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878220] [ 9979] 108 9979 6818 73 0 0 0 cleanup
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878223] [ 9981] 108 9981 6792 70 0 0 0 pickup
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878227] [10019] 108 10019 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878230] [10037] 108 10037 11108 158 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878233] [10042] 108 10042 11108 161 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878236] [10051] 108 10051 11108 157 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878239] [10238] 108 10238 6818 74 0 0 0 cleanup
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878242] [10371] 108 10371 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878245] [10375] 108 10375 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878248] [10381] 108 10381 11108 159 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878251] [10391] 108 10391 11108 159 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878254] [10392] 108 10392 11108 160 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878257] [10393] 108 10393 11108 159 1 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878260] [10394] 108 10394 11108 159 0 0 0 smtp
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878263] [11204] 1012 11204 944883 881686 0 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878266] [11594] 1012 11594 65535 3051 0 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878269] [11596] 1012 11596 11724 110 0 0 0 php
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878272] Out of memory: Kill process 11204 (php) score 920 or sacrifice child
Jul 16 17:01:37 ip-10-61-174-46 kernel: [1520992.878283] Killed process 11204 (php) total-vm:3779532kB, anon-rss:3526744kB, file-rss:0kB
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708240] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708245] php cpuset=/ mems_allowed=0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708248] Pid: 11992, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708251] Call Trace:
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708259] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708262] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708266] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708270] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708274] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708279] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708282] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708286] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708289] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708293] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708296] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708300] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708304] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708309] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708312] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708315] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708319] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708323] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708326] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708329] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708331] Mem-Info:
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708333] Node 0 DMA per-cpu:
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708336] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708337] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708339] Node 0 DMA32 per-cpu:
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708341] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708343] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708347] active_anon:930346 inactive_anon:1398 isolated_anon:0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708348] active_file:0 inactive_file:0 isolated_file:0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708349] unevictable:0 dirty:4 writeback:0 unstable:0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708350] free:4045 slab_reclaimable:2525 slab_unreclaimable:3267
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708350] mapped:1359 shmem:1409 pagetables:4610 bounce:0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708353] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708363] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708367] Node 0 DMA32 free:7844kB min:7848kB low:9808kB high:11772kB active_anon:3721384kB inactive_anon:5592kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:16kB writeback:0kB mapped:5436kB shmem:5636kB slab_reclaimable:10100kB slab_unreclaimable:13068kB kernel_stack:1096kB pagetables:18440kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2 all_unreclaimable? yes
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708376] lowmem_reserve[]: 0 0 0 0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708379] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708389] Node 0 DMA32: 220*4kB 75*8kB 47*16kB 15*32kB 11*64kB 4*128kB 5*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 7768kB
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708399] 1500 total pagecache pages
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708400] 0 pages in swap cache
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708402] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708403] Free swap = 0kB
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.708405] Total swap = 0kB
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714045] 985072 pages RAM
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714047] 24998 pages reserved
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714049] 4410 pages shared
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714050] 952639 pages non-shared
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714052] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714060] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714064] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714067] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714070] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714073] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714076] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714079] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714083] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714087] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714090] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714093] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714096] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714099] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714102] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714105] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714108] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714112] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714115] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714118] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714121] [ 6563] 0 6563 89584 2506 1 0 0 php5-fpm
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714125] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714128] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714131] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714134] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714137] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714140] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714143] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714146] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714149] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714152] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714155] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714159] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714162] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714165] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714168] [ 8228] 1012 8228 70415 5807 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714171] [ 8230] 1012 8230 69183 6210 1 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714174] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714177] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714180] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714183] [ 9132] 108 9132 6795 75 0 0 0 trivial-rewrite
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714186] [ 9133] 108 9133 11107 161 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714189] [ 9137] 108 9137 11107 159 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714192] [ 9183] 108 9183 6825 94 0 0 0 scache
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714195] [ 9189] 108 9189 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714198] [ 9199] 108 9199 11108 161 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714201] [ 9211] 108 9211 11108 159 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714204] [ 9217] 108 9217 11108 161 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714207] [ 9225] 108 9225 11108 161 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714210] [ 9239] 108 9239 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714213] [ 9259] 108 9259 11108 161 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714216] [ 9366] 108 9366 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714219] [ 9981] 108 9981 6792 70 0 0 0 pickup
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714222] [10019] 108 10019 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714225] [10037] 108 10037 11108 158 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714229] [10042] 108 10042 11108 161 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714232] [10051] 108 10051 11108 157 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714235] [10371] 108 10371 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714238] [10375] 108 10375 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714241] [10381] 108 10381 11108 159 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714244] [10391] 108 10391 11108 159 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714247] [10392] 108 10392 11108 160 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714250] [10393] 108 10393 11108 159 1 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714253] [10394] 108 10394 11108 159 0 0 0 smtp
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714256] [11666] 1012 11666 656801 593550 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714259] [11773] 1012 11773 323149 259861 1 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714262] [11880] 1012 11880 71457 8493 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714265] [11940] 1012 11940 1099 24 0 0 0 sh
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714268] [11941] 1012 11941 68825 5875 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714271] [11942] 1012 11942 71457 8466 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714274] [11957] 1012 11957 1099 25 0 0 0 sh
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714277] [11958] 1012 11958 4768 92 1 0 0 git
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714280] [11959] 1012 11959 68476 5523 1 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714283] [11973] 1012 11973 1099 24 0 0 0 sh
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714286] [11974] 1012 11974 10901 166 1 0 0 ssh
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714289] [11983] 1012 11983 46223 979 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714292] [11984] 1012 11984 43778 593 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714294] [11988] 1012 11988 12578 207 0 0 0 php
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714297] [11991] 1012 11991 40051 108 0 0 0 git
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714300] Out of memory: Kill process 11666 (php) score 619 or sacrifice child
Jul 16 17:01:59 ip-10-61-174-46 kernel: [1521014.714311] Killed process 11666 (php) total-vm:2627204kB, anon-rss:2374200kB, file-rss:0kB
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929829] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929833] php cpuset=/ mems_allowed=0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929836] Pid: 8230, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929837] Call Trace:
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929845] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929847] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929849] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929852] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929855] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929859] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929861] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929864] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929867] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929870] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929872] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929875] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929878] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929880] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929883] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929886] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929887] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929891] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929892] Mem-Info:
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929893] Node 0 DMA per-cpu:
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929895] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929897] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929898] Node 0 DMA32 per-cpu:
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929899] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929901] CPU 1: hi: 186, btch: 31 usd: 43
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929903] active_anon:932069 inactive_anon:1398 isolated_anon:0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929904] active_file:10 inactive_file:0 isolated_file:0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929904] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929905] free:4034 slab_reclaimable:2378 slab_unreclaimable:3028
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929906] mapped:1363 shmem:1409 pagetables:3611 bounce:0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929907] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929914] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929917] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3728276kB inactive_anon:5592kB active_file:40kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5452kB shmem:5636kB slab_reclaimable:9512kB slab_unreclaimable:12112kB kernel_stack:856kB pagetables:14444kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:158 all_unreclaimable? yes
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929923] lowmem_reserve[]: 0 0 0 0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929926] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929933] Node 0 DMA32: 146*4kB 430*8kB 102*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929940] 1428 total pagecache pages
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929941] 0 pages in swap cache
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929942] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929943] Free swap = 0kB
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.929944] Total swap = 0kB
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935531] 985072 pages RAM
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935533] 24998 pages reserved
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935535] 4379 pages shared
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935536] 952685 pages non-shared
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935538] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935546] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935550] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935553] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935556] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935559] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935562] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935565] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935569] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935573] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935576] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935579] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935583] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935586] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935589] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935592] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935595] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935599] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935602] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935605] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935608] [ 6563] 0 6563 89584 2526 1 0 0 php5-fpm
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935612] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935615] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935618] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935621] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935624] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935627] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935630] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935633] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935636] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935639] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935642] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935646] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935649] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935652] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935655] [ 8228] 1012 8228 70415 5812 0 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935658] [ 8230] 1012 8230 69183 6209 0 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935661] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935664] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935667] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935670] [ 9132] 108 9132 6795 75 0 0 0 trivial-rewrite
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935673] [ 9183] 108 9183 6825 94 1 0 0 scache
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935676] [ 9981] 108 9981 6792 76 0 0 0 pickup
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935679] [11773] 1012 11773 937907 874584 0 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935683] [12190] 1012 12190 71457 8466 1 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935686] [12216] 1012 12216 68579 5615 0 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935689] [12218] 1012 12218 31456 416 1 0 0 php
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935691] Out of memory: Kill process 11773 (php) score 912 or sacrifice child
Jul 16 17:02:16 ip-10-61-174-46 kernel: [1521031.935703] Killed process 11773 (php) total-vm:3751628kB, anon-rss:3498336kB, file-rss:0kB
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062878] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062883] php cpuset=/ mems_allowed=0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062887] Pid: 8252, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062889] Call Trace:
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062897] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062901] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062903] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062908] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062913] [<ffffffff81312aa6>] ? radix_tree_next_hole+0x36/0x50
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062917] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062921] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062924] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062928] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062932] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062935] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062938] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062941] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062943] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062946] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062948] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062952] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062955] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062958] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062960] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062962] Mem-Info:
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062963] Node 0 DMA per-cpu:
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062965] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062966] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062968] Node 0 DMA32 per-cpu:
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062969] CPU 0: hi: 186, btch: 31 usd: 18
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062971] CPU 1: hi: 186, btch: 31 usd: 64
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062973] active_anon:932115 inactive_anon:1398 isolated_anon:0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062974] active_file:0 inactive_file:4 isolated_file:7
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062974] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062975] free:4008 slab_reclaimable:2437 slab_unreclaimable:2996
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062976] mapped:1394 shmem:1409 pagetables:3542 bounce:0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062977] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062984] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062987] Node 0 DMA32 free:7696kB min:7848kB low:9808kB high:11772kB active_anon:3728460kB inactive_anon:5592kB active_file:0kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):28kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5576kB shmem:5636kB slab_reclaimable:9748kB slab_unreclaimable:11984kB kernel_stack:848kB pagetables:14168kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:134 all_unreclaimable? yes
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062993] lowmem_reserve[]: 0 0 0 0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.062996] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.063002] Node 0 DMA32: 358*4kB 125*8kB 103*16kB 41*32kB 4*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7696kB
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.063009] 1433 total pagecache pages
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.063010] 0 pages in swap cache
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.063012] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.063013] Free swap = 0kB
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.063014] Total swap = 0kB
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068527] 985072 pages RAM
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068529] 24998 pages reserved
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068530] 4400 pages shared
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068531] 952684 pages non-shared
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068533] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068542] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068546] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068549] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068552] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068555] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068559] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068562] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068565] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068569] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068572] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068575] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068579] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068582] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068585] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068588] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068591] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068594] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068597] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068600] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068604] [ 6563] 0 6563 89584 2510 0 0 0 php5-fpm
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068607] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068610] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068613] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068616] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068619] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068622] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068625] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068628] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068631] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068635] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068638] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068641] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068644] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068647] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068651] [ 8228] 1012 8228 70415 5833 0 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068654] [ 8230] 1012 8230 69183 6209 1 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068657] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068660] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068663] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068666] [ 9132] 108 9132 6795 75 0 0 0 trivial-rewrite
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068669] [ 9183] 108 9183 6825 94 1 0 0 scache
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068672] [ 9981] 108 9981 6792 76 0 0 0 pickup
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068676] [12363] 1012 12363 718373 655111 1 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068679] [12552] 1012 12552 288779 225530 0 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068682] [12695] 1012 12695 71457 8467 1 0 0 php
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068685] Out of memory: Kill process 12363 (php) score 683 or sacrifice child
Jul 16 17:02:46 ip-10-61-174-46 kernel: [1521062.068696] Killed process 12363 (php) total-vm:2873492kB, anon-rss:2620444kB, file-rss:0kB
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852106] ssh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852122] ssh cpuset=/ mems_allowed=0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852126] Pid: 13083, comm: ssh Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852128] Call Trace:
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852136] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852140] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852143] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852147] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852151] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852155] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852158] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852162] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852167] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852170] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852174] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852177] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852180] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852183] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852186] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852190] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852195] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852199] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852201] Mem-Info:
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852202] Node 0 DMA per-cpu:
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852205] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852207] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852208] Node 0 DMA32 per-cpu:
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852211] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852213] CPU 1: hi: 186, btch: 31 usd: 86
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852216] active_anon:931890 inactive_anon:1398 isolated_anon:0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852217] active_file:2 inactive_file:32 isolated_file:0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852218] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852219] free:4004 slab_reclaimable:2417 slab_unreclaimable:3052
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852220] mapped:1345 shmem:1409 pagetables:3678 bounce:0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852222] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852230] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852234] Node 0 DMA32 free:7680kB min:7848kB low:9808kB high:11772kB active_anon:3727560kB inactive_anon:5592kB active_file:8kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5380kB shmem:5636kB slab_reclaimable:9668kB slab_unreclaimable:12208kB kernel_stack:896kB pagetables:14712kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:309 all_unreclaimable? yes
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852243] lowmem_reserve[]: 0 0 0 0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852246] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852256] Node 0 DMA32: 460*4kB 78*8kB 42*16kB 46*32kB 6*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 7680kB
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852266] 1457 total pagecache pages
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852268] 0 pages in swap cache
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852270] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852271] Free swap = 0kB
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.852272] Total swap = 0kB
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857791] 985072 pages RAM
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857793] 24998 pages reserved
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857795] 4393 pages shared
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857796] 952652 pages non-shared
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857797] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857806] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857809] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857812] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857814] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857817] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857820] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857823] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857826] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857830] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857833] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857836] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857839] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857841] [ 983] 108 983 6865 119 0 0 0 qmgr
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857844] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857847] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857856] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857859] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857862] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857865] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857868] [ 6563] 0 6563 89584 2519 0 0 0 php5-fpm
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857872] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857875] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857878] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857881] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857884] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857887] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857890] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857893] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857895] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857898] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857901] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857905] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857908] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857911] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857914] [ 8228] 1012 8228 70415 5842 1 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857917] [ 8230] 1012 8230 69183 6210 1 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857920] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857922] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857926] [ 8253] 1012 8253 68618 5626 0 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857929] [ 9132] 108 9132 6795 75 1 0 0 trivial-rewrite
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857932] [ 9183] 108 9183 6825 94 1 0 0 scache
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857935] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857938] [12552] 1012 12552 932851 869502 0 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857941] [13065] 1012 13065 1099 24 0 0 0 sh
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857944] [13066] 1012 13066 68825 5876 0 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857947] [13074] 1012 13074 1099 26 0 0 0 sh
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857949] [13075] 1012 13075 4776 58 1 0 0 git
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857951] [13077] 1012 13077 68476 5523 1 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857953] [13080] 1012 13080 70593 7644 0 0 0 php
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857955] [13082] 1012 13082 1099 25 0 0 0 sh
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857957] [13083] 1012 13083 10901 166 1 0 0 ssh
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857959] Out of memory: Kill process 12552 (php) score 907 or sacrifice child
Jul 16 17:03:06 ip-10-61-174-46 kernel: [1521081.857968] Killed process 12552 (php) total-vm:3731404kB, anon-rss:3478004kB, file-rss:4kB
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812133] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812137] php cpuset=/ mems_allowed=0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812141] Pid: 26605, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812143] Call Trace:
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812151] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812154] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812157] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812161] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812167] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812170] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812174] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812177] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812181] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812184] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812187] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812191] [<ffffffff810062e9>] ? pte_mfn_to_pfn+0x19/0x110
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812197] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812200] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812203] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812206] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812209] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812212] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812216] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812220] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812223] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812226] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812228] Mem-Info:
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812230] Node 0 DMA per-cpu:
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812232] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812234] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812236] Node 0 DMA32 per-cpu:
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812238] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812240] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812243] active_anon:930849 inactive_anon:1398 isolated_anon:0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812244] active_file:1 inactive_file:0 isolated_file:0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812245] unevictable:0 dirty:8 writeback:9 unstable:0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812246] free:2841 slab_reclaimable:2445 slab_unreclaimable:3723
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812247] mapped:1367 shmem:1409 pagetables:4083 bounce:0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812249] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812257] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812261] Node 0 DMA32 free:3028kB min:7848kB low:9808kB high:11772kB active_anon:3723396kB inactive_anon:5592kB active_file:4kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:32kB writeback:36kB mapped:5468kB shmem:5636kB slab_reclaimable:9780kB slab_unreclaimable:14892kB kernel_stack:888kB pagetables:16332kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12 all_unreclaimable? yes
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812269] lowmem_reserve[]: 0 0 0 0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812272] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812282] Node 0 DMA32: 377*4kB 52*8kB 2*16kB 1*32kB 1*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 3076kB
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812291] 1417 total pagecache pages
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812293] 0 pages in swap cache
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812294] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812296] Free swap = 0kB
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.812297] Total swap = 0kB
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819582] 985072 pages RAM
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819584] 24998 pages reserved
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819586] 4372 pages shared
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819587] 953941 pages non-shared
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819588] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819597] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819601] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819604] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819607] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819610] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819613] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819616] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819619] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819623] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819626] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819629] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819632] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819634] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819637] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819640] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819643] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819646] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819649] [ 1242] 107 1242 9445 163 0 0 0 ntpd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819652] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819657] [ 6563] 0 6563 89584 2503 0 0 0 php5-fpm
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819660] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819663] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819666] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819669] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819671] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819674] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819676] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819679] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819682] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819685] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819687] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819690] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819693] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819696] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819699] [ 8228] 1012 8228 70863 6289 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819701] [ 8230] 1012 8230 69183 6222 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819704] [ 8232] 1012 8232 69859 6866 0 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819706] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819709] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819712] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819715] [26605] 1012 26605 338573 275336 0 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819718] [26607] 1012 26607 451348 388111 0 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819720] [26620] 1012 26620 71386 8438 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819723] [26630] 1012 26630 71386 8436 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819726] [26639] 1012 26639 140658 74177 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819729] [26658] 1012 26658 130032 60641 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819732] [26663] 1012 26663 93276 27699 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819734] [26665] 1012 26665 109122 40245 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819737] [26712] 1012 26712 1099 26 0 0 0 sh
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819740] [26713] 1012 26713 67296 4320 1 0 0 php
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819743] Out of memory: Kill process 26607 (php) score 405 or sacrifice child
Jul 16 17:12:41 ip-10-61-174-46 kernel: [1521656.819754] Killed process 26607 (php) total-vm:1805392kB, anon-rss:1552444kB, file-rss:0kB
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804701] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804706] php cpuset=/ mems_allowed=0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804709] Pid: 26723, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804711] Call Trace:
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804720] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804723] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804726] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804729] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804733] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804738] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804741] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804745] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804750] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804753] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804757] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804760] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804763] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804766] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804771] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804774] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804778] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804781] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804783] Mem-Info:
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804785] Node 0 DMA per-cpu:
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804787] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804789] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804791] Node 0 DMA32 per-cpu:
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804793] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804794] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804798] active_anon:931233 inactive_anon:1398 isolated_anon:0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804799] active_file:19 inactive_file:13 isolated_file:0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804800] unevictable:0 dirty:5 writeback:3 unstable:0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804801] free:2822 slab_reclaimable:2279 slab_unreclaimable:3611
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804802] mapped:1385 shmem:1409 pagetables:4139 bounce:0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804804] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804812] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804816] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724932kB inactive_anon:5592kB active_file:76kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:20kB writeback:12kB mapped:5540kB shmem:5636kB slab_reclaimable:9116kB slab_unreclaimable:14444kB kernel_stack:904kB pagetables:16556kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:222 all_unreclaimable? yes
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804825] lowmem_reserve[]: 0 0 0 0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804828] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804838] Node 0 DMA32: 167*4kB 117*8kB 5*16kB 0*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 3028kB
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804847] 1442 total pagecache pages
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804849] 0 pages in swap cache
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804851] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804852] Free swap = 0kB
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.804853] Total swap = 0kB
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810318] 985072 pages RAM
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810320] 24998 pages reserved
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810321] 4377 pages shared
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810322] 953946 pages non-shared
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810324] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810333] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810336] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810339] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810342] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810346] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810349] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810352] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810356] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810360] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810363] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810366] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810369] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810373] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810376] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810379] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810382] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810385] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810388] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810391] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810395] [ 6563] 0 6563 89584 2534 1 0 0 php5-fpm
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810398] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810401] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810404] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810407] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810410] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810413] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810416] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810419] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810422] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810425] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810428] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810432] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810435] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810438] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810441] [ 8228] 1012 8228 70863 6289 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810444] [ 8230] 1012 8230 69183 6222 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810447] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810450] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810453] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810456] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810460] [26605] 1012 26605 616542 553219 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810463] [26620] 1012 26620 71386 8442 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810466] [26630] 1012 26630 71386 8435 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810469] [26639] 1012 26639 200017 121038 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810472] [26658] 1012 26658 143978 79430 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810475] [26663] 1012 26663 116692 46555 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810479] [26665] 1012 26665 128886 59229 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810482] [26712] 1012 26712 1099 26 0 0 0 sh
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810485] [26713] 1012 26713 68825 5875 1 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810488] [26721] 1012 26721 1099 25 0 0 0 sh
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810491] [26722] 1012 26722 4655 57 1 0 0 git
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810494] [26723] 1012 26723 68434 5486 0 0 0 php
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810497] Out of memory: Kill process 26605 (php) score 577 or sacrifice child
Jul 16 17:12:48 ip-10-61-174-46 kernel: [1521663.810508] Killed process 26605 (php) total-vm:2466168kB, anon-rss:2212872kB, file-rss:4kB
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832135] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832140] php cpuset=/ mems_allowed=0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832143] Pid: 26658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832145] Call Trace:
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832153] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832156] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832159] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832163] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832167] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832171] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832177] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832181] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832185] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832188] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832191] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832195] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832202] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832206] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832209] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832212] Mem-Info:
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832213] Node 0 DMA per-cpu:
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832216] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832217] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832219] Node 0 DMA32 per-cpu:
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832221] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832223] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832226] active_anon:930353 inactive_anon:1398 isolated_anon:0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832227] active_file:24 inactive_file:1 isolated_file:0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832228] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832229] free:2915 slab_reclaimable:2276 slab_unreclaimable:3727
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832230] mapped:1356 shmem:1409 pagetables:4305 bounce:0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832232] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832240] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832244] Node 0 DMA32 free:3324kB min:7848kB low:9808kB high:11772kB active_anon:3721412kB inactive_anon:5592kB active_file:96kB inactive_file:4kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:9104kB slab_unreclaimable:14908kB kernel_stack:928kB pagetables:17220kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1569 all_unreclaimable? yes
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832253] lowmem_reserve[]: 0 0 0 0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832256] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832266] Node 0 DMA32: 421*4kB 1*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3324kB
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832275] 1434 total pagecache pages
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832277] 0 pages in swap cache
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832278] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832280] Free swap = 0kB
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.832281] Total swap = 0kB
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838063] 985072 pages RAM
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838066] 24998 pages reserved
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838067] 4384 pages shared
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838068] 953862 pages non-shared
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838070] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838079] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838082] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838085] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838088] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838092] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838095] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838098] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838101] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838105] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838108] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838111] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838115] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838118] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838121] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838124] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838127] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838130] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838133] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838136] [15657] 0 15657 4777 59 1 0 0 cron
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838140] [ 6563] 0 6563 89584 2518 1 0 0 php5-fpm
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838143] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838146] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838149] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838152] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838155] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838158] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838161] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838164] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838167] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838170] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838173] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838177] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838180] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838183] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838186] [ 8228] 1012 8228 70863 6290 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838189] [ 8230] 1012 8230 69183 6222 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838192] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838195] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838198] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838201] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838204] [26620] 1012 26620 71450 8443 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838207] [26630] 1012 26630 71386 8439 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838210] [26639] 1012 26639 489112 425822 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838213] [26658] 1012 26658 273676 210413 0 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838216] [26663] 1012 26663 139142 72204 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838219] [26665] 1012 26665 223570 141075 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838222] [26746] 1012 26746 1099 25 0 0 0 sh
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838225] [26747] 1012 26747 67752 4790 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838228] [26748] 1012 26748 1099 25 0 0 0 sh
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838231] [26749] 1012 26749 67687 4755 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838234] [26750] 1012 26750 1099 25 0 0 0 sh
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838237] [26751] 1012 26751 67753 4832 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838240] [26752] 1012 26752 1099 24 1 0 0 sh
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838243] [26753] 1012 26753 67817 4853 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838246] [26760] 1012 26760 63863 1195 1 0 0 php
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838249] Out of memory: Kill process 26639 (php) score 444 or sacrifice child
Jul 16 17:12:59 ip-10-61-174-46 kernel: [1521674.838259] Killed process 26639 (php) total-vm:1956448kB, anon-rss:1703288kB, file-rss:0kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812226] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812230] php cpuset=/ mems_allowed=0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812233] Pid: 26663, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812234] Call Trace:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812241] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812243] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812246] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812249] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812254] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812257] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812260] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812262] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812265] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812268] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812270] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812273] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812276] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812278] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812282] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812284] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812286] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812289] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812291] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812293] Mem-Info:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812294] Node 0 DMA per-cpu:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812296] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812297] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812299] Node 0 DMA32 per-cpu:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812300] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812302] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812304] active_anon:930245 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812305] active_file:8 inactive_file:6 isolated_file:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812306] unevictable:0 dirty:13 writeback:2 unstable:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812306] free:2844 slab_reclaimable:2436 slab_unreclaimable:3692
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812307] mapped:1370 shmem:1409 pagetables:4419 bounce:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812308] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812315] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812318] Node 0 DMA32 free:3040kB min:7848kB low:9808kB high:11772kB active_anon:3720980kB inactive_anon:5592kB active_file:32kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:52kB writeback:8kB mapped:5480kB shmem:5636kB slab_reclaimable:9744kB slab_unreclaimable:14768kB kernel_stack:960kB pagetables:17676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3216 all_unreclaimable? yes
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812325] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812327] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812334] Node 0 DMA32: 150*4kB 48*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 3128kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812341] 1437 total pagecache pages
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812342] 0 pages in swap cache
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812343] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812344] Free swap = 0kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.812345] Total swap = 0kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817746] 985072 pages RAM
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817749] 24998 pages reserved
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817750] 4384 pages shared
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817751] 953905 pages non-shared
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817754] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817762] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817766] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817769] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817772] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817776] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817779] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817782] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817786] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817790] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817793] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817796] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817799] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817802] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817805] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817809] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817812] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817815] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817818] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817821] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817825] [ 6563] 0 6563 89584 2502 0 0 0 php5-fpm
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817828] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817831] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817834] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817837] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817840] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817843] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817846] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817850] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817853] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817856] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817859] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817862] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817865] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817868] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817871] [ 8228] 1012 8228 70863 6289 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817874] [ 8230] 1012 8230 69242 6231 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817877] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817880] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817883] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817887] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817890] [26620] 1012 26620 71450 8449 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817893] [26630] 1012 26630 90594 26661 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817896] [26658] 1012 26658 378770 315456 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817899] [26663] 1012 26663 284545 221254 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817903] [26665] 1012 26665 329551 266277 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817906] [26748] 1012 26748 1099 25 0 0 0 sh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817909] [26749] 1012 26749 68825 5880 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817911] [26760] 1012 26760 73973 10950 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817915] [26762] 1012 26762 71311 8366 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817918] [26772] 1012 26772 1099 25 0 0 0 sh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817921] [26773] 1012 26773 68825 5852 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817924] [26777] 1012 26777 1099 26 1 0 0 sh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817927] [26778] 1012 26778 1099 25 1 0 0 sh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817930] [26779] 1012 26779 68760 5800 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817933] [26780] 1012 26780 5216 86 1 0 0 git
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817935] [26781] 1012 26781 68476 5522 0 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817939] [26783] 1012 26783 1099 24 1 0 0 sh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817942] [26784] 1012 26784 68760 5799 1 0 0 php
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817945] [26793] 1012 26793 1099 25 0 0 0 sh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817948] [26794] 1012 26794 10901 166 1 0 0 ssh
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817951] [26795] 1012 26795 9813 111 0 0 0 git
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817953] Out of memory: Kill process 26658 (php) score 329 or sacrifice child
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.817965] Killed process 26658 (php) total-vm:1515080kB, anon-rss:1261824kB, file-rss:0kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916078] php: page allocation failure: order:0, mode:0x280da
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916083] Pid: 26658, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916085] Call Trace:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916093] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916098] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916102] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916106] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916110] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916114] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916117] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916121] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916127] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916130] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916134] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916138] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916141] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916145] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916147] Mem-Info:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916148] Node 0 DMA per-cpu:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916151] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916153] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916154] Node 0 DMA32 per-cpu:
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916156] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916158] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916162] active_anon:930266 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916163] active_file:0 inactive_file:25 isolated_file:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916164] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916164] free:2826 slab_reclaimable:2418 slab_unreclaimable:3692
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916166] mapped:1356 shmem:1409 pagetables:4431 bounce:0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916169] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916177] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916181] Node 0 DMA32 free:2968kB min:7848kB low:9808kB high:11772kB active_anon:3721064kB inactive_anon:5592kB active_file:0kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:9672kB slab_unreclaimable:14768kB kernel_stack:976kB pagetables:17724kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:18 all_unreclaimable? no
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916190] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916193] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916203] Node 0 DMA32: 150*4kB 44*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2968kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916214] 1434 total pagecache pages
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916215] 0 pages in swap cache
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916217] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916219] Free swap = 0kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.916220] Total swap = 0kB
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.923437] 985072 pages RAM
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.923440] 24998 pages reserved
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.923441] 4384 pages shared
Jul 16 17:13:05 ip-10-61-174-46 kernel: [1521680.923442] 953945 pages non-shared
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896151] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896155] php cpuset=/ mems_allowed=0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896158] Pid: 26760, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896160] Call Trace:
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896167] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896169] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896172] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896174] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896178] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896181] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896183] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896186] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896189] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896192] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896194] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896197] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896201] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896204] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896208] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896211] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896212] Mem-Info:
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896214] Node 0 DMA per-cpu:
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896215] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896217] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896218] Node 0 DMA32 per-cpu:
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896219] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896221] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896223] active_anon:931234 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896224] active_file:25 inactive_file:27 isolated_file:0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896225] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896225] free:3935 slab_reclaimable:2481 slab_unreclaimable:3285
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896226] mapped:1353 shmem:1409 pagetables:3911 bounce:0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896228] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896234] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896237] Node 0 DMA32 free:7404kB min:7848kB low:9808kB high:11772kB active_anon:3724936kB inactive_anon:5592kB active_file:100kB inactive_file:108kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5636kB slab_reclaimable:9924kB slab_unreclaimable:13140kB kernel_stack:864kB pagetables:15644kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:24948 all_unreclaimable? yes
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896244] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896246] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896253] Node 0 DMA32: 606*4kB 254*8kB 53*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7416kB
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896259] 1437 total pagecache pages
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896261] 0 pages in swap cache
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896262] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896263] Free swap = 0kB
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.896264] Total swap = 0kB
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901835] 985072 pages RAM
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901837] 24998 pages reserved
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901839] 4387 pages shared
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901840] 952844 pages non-shared
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901842] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901851] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901854] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901858] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901866] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901870] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901873] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901876] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901880] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901884] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901887] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901890] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901893] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901897] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901900] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901903] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901906] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901909] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901912] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901915] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901919] [ 6563] 0 6563 89584 2537 0 0 0 php5-fpm
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901922] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901925] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901929] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901932] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901935] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901938] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901941] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901944] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901947] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901950] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901953] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901956] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901959] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901962] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901965] [ 8228] 1012 8228 70863 6289 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901969] [ 8230] 1012 8230 69242 6232 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901972] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901974] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901977] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901981] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901984] [26620] 1012 26620 71450 8453 0 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901987] [26630] 1012 26630 225194 138026 0 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901990] [26663] 1012 26663 417993 354740 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901993] [26665] 1012 26665 413716 350460 0 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901996] [26760] 1012 26760 80324 17325 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.901999] [26762] 1012 26762 73363 10413 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.902002] [26813] 1012 26813 71386 8427 1 0 0 php
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.902005] Out of memory: Kill process 26663 (php) score 370 or sacrifice child
Jul 16 17:13:11 ip-10-61-174-46 kernel: [1521686.902016] Killed process 26663 (php) total-vm:1671972kB, anon-rss:1418960kB, file-rss:0kB
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564142] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564146] php cpuset=/ mems_allowed=0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564149] Pid: 26760, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564151] Call Trace:
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564158] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564162] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564165] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564168] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564172] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564174] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564177] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564181] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564184] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564187] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564190] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564192] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564195] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564199] [<ffffffff81090448>] ? up_read+0x18/0x30
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564202] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564205] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564208] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564210] Mem-Info:
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564211] Node 0 DMA per-cpu:
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564213] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564214] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564215] Node 0 DMA32 per-cpu:
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564217] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564218] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564221] active_anon:930922 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564222] active_file:12 inactive_file:28 isolated_file:0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564222] unevictable:0 dirty:20 writeback:0 unstable:0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564223] free:3035 slab_reclaimable:2435 slab_unreclaimable:3504
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564224] mapped:1389 shmem:1409 pagetables:3799 bounce:0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564225] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564232] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564235] Node 0 DMA32 free:3804kB min:7848kB low:9808kB high:11772kB active_anon:3723688kB inactive_anon:5592kB active_file:48kB inactive_file:112kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:80kB writeback:0kB mapped:5556kB shmem:5636kB slab_reclaimable:9740kB slab_unreclaimable:14016kB kernel_stack:856kB pagetables:15196kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2249 all_unreclaimable? yes
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564241] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564244] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564250] Node 0 DMA32: 603*4kB 0*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 3804kB
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564257] 1422 total pagecache pages
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564258] 0 pages in swap cache
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564259] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564260] Free swap = 0kB
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.564261] Total swap = 0kB
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569968] 985072 pages RAM
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569970] 24998 pages reserved
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569971] 4378 pages shared
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569972] 953726 pages non-shared
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569974] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569983] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569987] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569990] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569993] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569996] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.569999] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570003] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570006] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570010] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570013] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570016] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570020] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570024] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570027] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570030] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570034] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570037] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570040] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570043] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570047] [ 6563] 0 6563 89584 2521 0 0 0 php5-fpm
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570050] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570053] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570056] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570059] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570062] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570065] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570068] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570071] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570074] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570078] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570080] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570084] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570087] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570090] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570093] [ 8228] 1012 8228 70863 6290 0 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570096] [ 8230] 1012 8230 69242 6233 1 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570099] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570102] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570105] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570108] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570112] [26620] 1012 26620 71450 8455 0 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570115] [26630] 1012 26630 383440 320146 0 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570118] [26665] 1012 26665 525018 461709 1 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570121] [26760] 1012 26760 85092 21828 0 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570124] [26762] 1012 26762 135754 66860 1 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570127] [26813] 1012 26813 71386 8436 1 0 0 php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570130] [26887] 1012 26887 1469 19 1 0 0 exec_daemon.php
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570133] Out of memory: Kill process 26665 (php) score 481 or sacrifice child
Jul 16 17:13:16 ip-10-61-174-46 kernel: [1521691.570145] Killed process 26665 (php) total-vm:2100072kB, anon-rss:1846832kB, file-rss:4kB
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828128] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828133] ntpd cpuset=/ mems_allowed=0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828136] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828138] Call Trace:
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828145] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828148] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828150] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828153] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828156] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828160] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828162] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828165] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828167] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828171] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828173] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828176] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828179] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828183] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828185] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828188] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828191] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828192] Mem-Info:
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828193] Node 0 DMA per-cpu:
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828195] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828197] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828198] Node 0 DMA32 per-cpu:
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828199] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828201] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828203] active_anon:930901 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828204] active_file:0 inactive_file:26 isolated_file:0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828204] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828205] free:2822 slab_reclaimable:2543 slab_unreclaimable:3502
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828206] mapped:1356 shmem:1409 pagetables:4081 bounce:0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828207] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828214] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828217] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3723604kB inactive_anon:5592kB active_file:0kB inactive_file:104kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:10172kB slab_unreclaimable:14008kB kernel_stack:896kB pagetables:16324kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1002 all_unreclaimable? yes
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828223] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828226] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828233] Node 0 DMA32: 216*4kB 9*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2952kB
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828239] 1427 total pagecache pages
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828240] 0 pages in swap cache
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828242] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828243] Free swap = 0kB
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.828244] Total swap = 0kB
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833723] 985072 pages RAM
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833725] 24998 pages reserved
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833726] 4378 pages shared
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833727] 953965 pages non-shared
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833729] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833738] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833742] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833745] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833748] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833751] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833754] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833757] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833761] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833765] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833768] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833771] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833775] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833778] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833781] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833784] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833787] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833790] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833793] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833796] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833800] [ 6563] 0 6563 89584 2505 0 0 0 php5-fpm
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833803] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833806] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833809] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833812] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833815] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833819] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833822] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833825] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833828] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833831] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833834] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833837] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833840] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833843] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833846] [ 8228] 1012 8228 70863 6290 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833849] [ 8230] 1012 8230 69242 6232 0 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833853] [ 8232] 1012 8232 69859 6866 0 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833856] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833858] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833862] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833866] [26620] 1012 26620 138714 70226 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833869] [26630] 1012 26630 527513 464253 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833872] [26760] 1012 26760 110026 40876 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833875] [26762] 1012 26762 337805 274553 0 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833878] [26813] 1012 26813 71450 8460 0 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833881] [26887] 1012 26887 71386 8427 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833884] [26893] 1012 26893 1099 25 0 0 0 sh
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833887] [26894] 1012 26894 69250 6275 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833890] [26896] 1012 26896 1099 25 0 0 0 sh
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833893] [26898] 1012 26898 69046 6042 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833896] [26934] 1012 26934 71322 8344 1 0 0 php
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833899] Out of memory: Kill process 26630 (php) score 484 or sacrifice child
Jul 16 17:13:27 ip-10-61-174-46 kernel: [1521702.833911] Killed process 26630 (php) total-vm:2110052kB, anon-rss:1857012kB, file-rss:0kB
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320504] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320509] php cpuset=/ mems_allowed=0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320512] Pid: 26887, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320514] Call Trace:
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320522] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320525] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320528] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320532] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320537] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320541] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320544] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320548] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320551] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320554] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320558] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320562] [<ffffffff81144d28>] ? do_mmap_pgoff+0x348/0x360
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320565] [<ffffffff81144e95>] ? sys_mmap_pgoff+0x155/0x230
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320570] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320572] Mem-Info:
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320573] Node 0 DMA per-cpu:
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320576] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320578] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320579] Node 0 DMA32 per-cpu:
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320581] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320582] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320585] active_anon:929665 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320585] active_file:0 inactive_file:8 isolated_file:0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320586] unevictable:0 dirty:21 writeback:0 unstable:0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320587] free:2822 slab_reclaimable:2430 slab_unreclaimable:3669
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320587] mapped:1369 shmem:1409 pagetables:4511 bounce:0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320589] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320595] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320598] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3718660kB inactive_anon:5592kB active_file:0kB inactive_file:32kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:84kB writeback:0kB mapped:5476kB shmem:5636kB slab_reclaimable:9720kB slab_unreclaimable:14676kB kernel_stack:992kB pagetables:18044kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:48 all_unreclaimable? yes
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320605] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320607] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320614] Node 0 DMA32: 338*4kB 0*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3032kB
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320621] 1422 total pagecache pages
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320622] 0 pages in swap cache
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320623] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320625] Free swap = 0kB
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.320625] Total swap = 0kB
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326202] 985072 pages RAM
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326205] 24998 pages reserved
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326206] 4387 pages shared
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326207] 953924 pages non-shared
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326209] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326218] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326222] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326225] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326228] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326231] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326234] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326237] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326241] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326245] [ 790] 101 790 63477 662 1 0 0 rsyslogd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326248] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326251] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326254] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326257] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326260] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326264] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326267] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326270] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326273] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326276] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326279] [ 6563] 0 6563 89584 2489 1 0 0 php5-fpm
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326283] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326286] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326289] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326292] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326295] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326298] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326301] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326304] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326307] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326310] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326313] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326316] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326319] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326322] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326325] [ 8228] 1012 8228 70863 6290 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326329] [ 8230] 1012 8230 69242 6235 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326332] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326335] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326338] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326341] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326344] [26620] 1012 26620 360206 296899 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326348] [26760] 1012 26760 140658 74103 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326351] [26762] 1012 26762 433939 370676 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326354] [26813] 1012 26813 71450 8462 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326357] [26887] 1012 26887 114140 44875 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326360] [26934] 1012 26934 115648 46126 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326363] [26999] 1012 26999 1099 24 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326366] [27000] 1012 27000 1099 25 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326369] [27001] 1012 27001 68982 6020 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326372] [27002] 1012 27002 76875 13907 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326374] [27003] 1012 27003 52273 1537 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326377] [27013] 1012 27013 1099 25 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326380] [27014] 1012 27014 68976 6041 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326383] [27016] 1012 27016 1099 25 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326387] [27017] 1012 27017 68825 5877 1 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326390] [27055] 1012 27055 1099 25 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326393] [27056] 1012 27056 5216 68 0 0 0 git
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326396] [27059] 1012 27059 68476 5522 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326399] [27070] 1012 27070 68656 5670 0 0 0 php
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326402] [27076] 1012 27076 1099 25 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326405] [27077] 1012 27077 10901 174 0 0 0 ssh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326408] [27108] 1012 27108 1099 26 0 0 0 sh
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326411] [27109] 1012 27109 4684 56 1 0 0 git
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326413] Out of memory: Kill process 26762 (php) score 386 or sacrifice child
Jul 16 17:13:34 ip-10-61-174-46 kernel: [1521710.326425] Killed process 26762 (php) total-vm:1735756kB, anon-rss:1482704kB, file-rss:0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740063] swapper/1: page allocation failure: order:0, mode:0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740068] Pid: 0, comm: swapper/1 Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740069] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740071] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740080] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740084] [<ffffffff8104f504>] ? __enqueue_entity+0x74/0x80
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740088] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740091] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740093] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740098] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740102] [<ffffffff81319f51>] ? csum_partial+0x11/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740106] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740109] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740111] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740113] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740116] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740120] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740123] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740125] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740129] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740132] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740134] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740137] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740139] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740141] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740143] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740145] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740147] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740149] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740152] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740154] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740157] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740159] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740161] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740164] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740166] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740169] [<ffffffff8109ccdd>] ? tick_do_update_jiffies64+0x9d/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740173] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740176] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740178] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740181] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740184] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740185] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740191] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740193] [<ffffffff8100a570>] ? xen_safe_halt+0x10/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740196] [<ffffffff8101cd43>] ? default_idle+0x53/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740198] [<ffffffff81013236>] ? cpu_idle+0xd6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740201] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740204] [<ffffffff816375a4>] ? cpu_bringup_and_idle+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740206] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740207] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740208] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740210] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740211] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740212] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740214] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740216] active_anon:930400 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740217] active_file:6 inactive_file:14 isolated_file:27
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740218] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740218] free:2817 slab_reclaimable:2432 slab_unreclaimable:3467
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740219] mapped:1347 shmem:1409 pagetables:4548 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740220] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740227] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740230] Node 0 DMA32 free:2932kB min:7848kB low:9808kB high:11772kB active_anon:3721600kB inactive_anon:5592kB active_file:24kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5636kB slab_reclaimable:9728kB slab_unreclaimable:13868kB kernel_stack:1040kB pagetables:18192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740236] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740239] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740245] Node 0 DMA32: 195*4kB 19*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2948kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740252] 1431 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740253] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740254] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740256] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740256] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740258] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740260] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.740262] node 0: slabs: 239, objs: 3824, free: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744061] swapper/1: page allocation failure: order:0, mode:0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744063] Pid: 0, comm: swapper/1 Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744064] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744065] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744070] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744072] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744075] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744077] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744080] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744083] [<ffffffff81319f51>] ? csum_partial+0x11/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744085] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744087] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744090] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744092] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744094] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744096] [<ffffffff8105151d>] ? ttwu_do_wakeup+0x3d/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744098] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744101] [<ffffffff81053896>] ? ttwu_do_activate.constprop.178+0x66/0x70
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744103] [<ffffffff81053956>] ? ttwu_queue+0xb6/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744105] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744108] [<ffffffff8106071e>] ? try_to_wake_up+0x18e/0x200
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744110] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744113] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744115] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744117] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744119] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744122] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744124] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744126] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744128] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744130] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744132] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744133] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744136] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744138] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744140] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744142] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744144] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744147] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744149] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744151] [<ffffffff8109ccdd>] ? tick_do_update_jiffies64+0x9d/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744154] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744156] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744158] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744160] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744163] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744164] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744168] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744171] [<ffffffff8100a570>] ? xen_safe_halt+0x10/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744173] [<ffffffff8101cd43>] ? default_idle+0x53/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744175] [<ffffffff81013236>] ? cpu_idle+0xd6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744177] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744180] [<ffffffff816375a4>] ? cpu_bringup_and_idle+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744181] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744182] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744184] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744185] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744186] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744187] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744189] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744191] active_anon:930400 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744192] active_file:6 inactive_file:14 isolated_file:27
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744192] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744193] free:2817 slab_reclaimable:2432 slab_unreclaimable:3467
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744194] mapped:1347 shmem:1409 pagetables:4548 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744195] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744201] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744204] Node 0 DMA32 free:2932kB min:7848kB low:9808kB high:11772kB active_anon:3721600kB inactive_anon:5592kB active_file:24kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5636kB slab_reclaimable:9728kB slab_unreclaimable:13868kB kernel_stack:1040kB pagetables:18192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744210] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744212] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744218] Node 0 DMA32: 195*4kB 19*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2948kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744225] 1431 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744226] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744227] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744228] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744229] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744230] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744232] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.744233] node 0: slabs: 239, objs: 3824, free: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756061] swapper/1: page allocation failure: order:0, mode:0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756063] Pid: 0, comm: swapper/1 Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756064] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756065] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756070] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756073] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756076] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756078] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756081] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756084] [<ffffffff81319f51>] ? csum_partial+0x11/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756086] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756088] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756091] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756093] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756095] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756097] [<ffffffff8105151d>] ? ttwu_do_wakeup+0x3d/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756099] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756101] [<ffffffff81053896>] ? ttwu_do_activate.constprop.178+0x66/0x70
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756103] [<ffffffff81053956>] ? ttwu_queue+0xb6/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756106] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756108] [<ffffffff8106071e>] ? try_to_wake_up+0x18e/0x200
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756110] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756113] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756115] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756117] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756119] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756122] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756123] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756125] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756127] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756129] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756131] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756133] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756135] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756137] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756140] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756142] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756144] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756147] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756149] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756151] [<ffffffff8109ccdd>] ? tick_do_update_jiffies64+0x9d/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756154] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756156] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756158] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756160] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756163] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756164] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756168] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756171] [<ffffffff8100a570>] ? xen_safe_halt+0x10/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756173] [<ffffffff8101cd43>] ? default_idle+0x53/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756175] [<ffffffff81013236>] ? cpu_idle+0xd6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756177] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756180] [<ffffffff816375a4>] ? cpu_bringup_and_idle+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756181] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756182] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756183] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756184] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756185] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756187] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756188] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756190] active_anon:930400 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756191] active_file:6 inactive_file:14 isolated_file:27
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756192] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756192] free:2817 slab_reclaimable:2432 slab_unreclaimable:3467
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756193] mapped:1347 shmem:1409 pagetables:4548 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756195] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756200] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756203] Node 0 DMA32 free:2932kB min:7848kB low:9808kB high:11772kB active_anon:3721600kB inactive_anon:5592kB active_file:24kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5636kB slab_reclaimable:9728kB slab_unreclaimable:13868kB kernel_stack:1040kB pagetables:18192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756209] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756211] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756218] Node 0 DMA32: 195*4kB 19*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2948kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756224] 1431 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756225] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756226] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756227] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756228] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756229] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756231] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756232] node 0: slabs: 239, objs: 3824, free: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756237] swapper/1: page allocation failure: order:0, mode:0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756239] Pid: 0, comm: swapper/1 Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756240] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756241] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756245] [<ffffffff8164a33f>] ? slab_out_of_memory+0xfc/0x115
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756248] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756250] [<ffffffff8105151d>] ? ttwu_do_wakeup+0x3d/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756253] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756255] [<ffffffff81053896>] ? ttwu_do_activate.constprop.178+0x66/0x70
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756258] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756261] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756263] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756266] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756268] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756270] [<ffffffff8157e28b>] ? ip_finish_output+0x16b/0x2f0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756273] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756274] [<ffffffff8157e630>] ? ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756277] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756279] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756280] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756282] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756285] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756287] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756289] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756291] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756293] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756295] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756297] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756299] [<ffffffff8109ccdd>] ? tick_do_update_jiffies64+0x9d/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756302] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756304] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756306] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756308] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756311] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756312] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756317] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756319] [<ffffffff8100a570>] ? xen_safe_halt+0x10/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756321] [<ffffffff8101cd43>] ? default_idle+0x53/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756323] [<ffffffff81013236>] ? cpu_idle+0xd6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756325] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756328] [<ffffffff816375a4>] ? cpu_bringup_and_idle+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756329] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756330] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756331] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756332] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756333] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756335] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756336] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756338] active_anon:930400 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756339] active_file:6 inactive_file:14 isolated_file:27
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756339] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756340] free:2817 slab_reclaimable:2432 slab_unreclaimable:3467
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756341] mapped:1347 shmem:1409 pagetables:4548 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756342] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756348] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756350] Node 0 DMA32 free:2932kB min:7848kB low:9808kB high:11772kB active_anon:3721600kB inactive_anon:5592kB active_file:24kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5636kB slab_reclaimable:9728kB slab_unreclaimable:13868kB kernel_stack:1040kB pagetables:18192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756356] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756359] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756365] Node 0 DMA32: 195*4kB 19*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2948kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756371] 1431 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756372] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756373] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756374] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756375] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756376] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756378] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.756380] node 0: slabs: 239, objs: 3824, free: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768061] swapper/1: page allocation failure: order:0, mode:0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768063] Pid: 0, comm: swapper/1 Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768064] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768065] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768069] [<ffffffff8111e4b3>] ? show_free_areas+0x7c3/0x7e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768071] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768074] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768076] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768079] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768082] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768084] [<ffffffff810d98e9>] ? handle_irq_event+0x59/0x80
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768086] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768088] [<ffffffff810dc924>] ? handle_edge_irq+0x94/0x130
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768091] [<ffffffff813ab2a7>] ? __xen_evtchn_do_upcall+0x2a7/0x310
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768093] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768096] [<ffffffff810e119f>] ? rcu_enter_nohz+0x4f/0x80
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768098] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768100] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768102] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768105] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768107] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768110] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768112] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768114] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768116] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768118] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768120] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768122] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768124] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768126] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768128] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768130] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768132] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768134] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768137] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768139] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768141] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768143] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768145] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768147] [<ffffffff8109ccdd>] ? tick_do_update_jiffies64+0x9d/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768150] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768153] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768155] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768157] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768159] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768161] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768165] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768167] [<ffffffff8100a570>] ? xen_safe_halt+0x10/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768169] [<ffffffff8101cd43>] ? default_idle+0x53/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768172] [<ffffffff81013236>] ? cpu_idle+0xd6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768174] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768176] [<ffffffff816375a4>] ? cpu_bringup_and_idle+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768178] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768178] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768180] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768181] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768182] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768183] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768185] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768187] active_anon:930400 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768188] active_file:6 inactive_file:14 isolated_file:27
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768188] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768189] free:2817 slab_reclaimable:2432 slab_unreclaimable:3467
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768190] mapped:1347 shmem:1409 pagetables:4548 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768191] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768197] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768199] Node 0 DMA32 free:2932kB min:7848kB low:9808kB high:11772kB active_anon:3721600kB inactive_anon:5592kB active_file:24kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5636kB slab_reclaimable:9728kB slab_unreclaimable:13868kB kernel_stack:1040kB pagetables:18192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768205] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768208] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768214] Node 0 DMA32: 195*4kB 19*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2948kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768220] 1431 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768221] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768222] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768223] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768224] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768225] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768227] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.768228] node 0: slabs: 239, objs: 3824, free: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772059] swapper/1: page allocation failure: order:0, mode:0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772061] Pid: 0, comm: swapper/1 Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772062] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772063] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772067] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772069] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772072] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772074] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772077] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772079] [<ffffffff810d98e9>] ? handle_irq_event+0x59/0x80
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772081] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772083] [<ffffffff810dc924>] ? handle_edge_irq+0x94/0x130
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772086] [<ffffffff813ab2a7>] ? __xen_evtchn_do_upcall+0x2a7/0x310
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772088] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772091] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772092] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772095] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772097] [<ffffffff8100122a>] ? hypercall_page+0x22a/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772100] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772102] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772104] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772106] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772108] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772110] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772113] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772115] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772117] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772119] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772120] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772122] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772124] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772127] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772128] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772131] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772133] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772135] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772137] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772139] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772142] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772144] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772146] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772148] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772151] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772152] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772156] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772159] [<ffffffff8100a570>] ? xen_safe_halt+0x10/0x20
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772161] [<ffffffff8101cd43>] ? default_idle+0x53/0x1d0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772163] [<ffffffff81013236>] ? cpu_idle+0xd6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772165] [<ffffffff8100ad49>] ? xen_irq_enable_direct_reloc+0x4/0x4
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772167] [<ffffffff816375a4>] ? cpu_bringup_and_idle+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772169] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772170] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772171] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772172] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772173] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772175] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772176] CPU 1: hi: 186, btch: 31 usd: 34
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772178] active_anon:930400 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772179] active_file:6 inactive_file:14 isolated_file:27
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772179] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772180] free:2817 slab_reclaimable:2432 slab_unreclaimable:3467
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772181] mapped:1347 shmem:1409 pagetables:4548 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772182] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772188] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772190] Node 0 DMA32 free:2932kB min:7848kB low:9808kB high:11772kB active_anon:3721600kB inactive_anon:5592kB active_file:24kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):108kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5388kB shmem:5636kB slab_reclaimable:9728kB slab_unreclaimable:13868kB kernel_stack:1040kB pagetables:18192kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772196] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772199] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772205] Node 0 DMA32: 195*4kB 19*8kB 0*16kB 1*32kB 1*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2948kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772211] 1431 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772212] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772213] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772214] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772215] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772216] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772218] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521717.772219] node 0: slabs: 239, objs: 3824, free: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808138] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808143] php cpuset=/ mems_allowed=0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808147] Pid: 27002, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808149] Call Trace:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808171] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808176] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808182] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808187] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808191] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808195] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808201] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808204] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808207] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808211] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808215] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808219] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808221] Mem-Info:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808222] Node 0 DMA per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808225] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808227] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808228] Node 0 DMA32 per-cpu:
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808231] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808233] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808236] active_anon:930437 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808237] active_file:5 inactive_file:28 isolated_file:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808238] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808239] free:2822 slab_reclaimable:2408 slab_unreclaimable:3507
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808239] mapped:1358 shmem:1409 pagetables:4552 bounce:0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808250] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808254] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3721748kB inactive_anon:5592kB active_file:20kB inactive_file:112kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5432kB shmem:5636kB slab_reclaimable:9632kB slab_unreclaimable:14028kB kernel_stack:1000kB pagetables:18208kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:692 all_unreclaimable? yes
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808262] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808266] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808276] Node 0 DMA32: 240*4kB 17*8kB 0*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3048kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808285] 1430 total pagecache pages
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808287] 0 pages in swap cache
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808288] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808290] Free swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.808291] Total swap = 0kB
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813842] 985072 pages RAM
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813844] 24998 pages reserved
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813845] 4380 pages shared
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813846] 953927 pages non-shared
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813847] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813855] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813858] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813860] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813862] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813864] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813867] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813869] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813871] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813875] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813877] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813879] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813881] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813883] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813886] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813888] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813890] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813893] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813895] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813897] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813899] [ 6563] 0 6563 89584 2528 0 0 0 php5-fpm
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813902] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813904] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813907] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813909] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813911] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813913] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813915] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813917] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813920] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813922] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813924] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813926] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813928] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813930] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813933] [ 8228] 1012 8228 70863 6290 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813935] [ 8230] 1012 8230 69242 6236 1 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813937] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813939] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813941] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813943] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813946] [26620] 1012 26620 606045 542818 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813949] [26760] 1012 26760 149507 86025 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813951] [26813] 1012 26813 71450 8462 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813953] [26887] 1012 26887 144382 78904 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813955] [26934] 1012 26934 145406 80089 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813957] [27002] 1012 27002 116800 47343 1 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813959] [27070] 1012 27070 71386 8379 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813961] [27116] 1012 27116 1099 24 1 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813963] [27117] 1012 27117 68825 5876 1 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813965] [27131] 1012 27131 1099 25 0 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813967] [27132] 1012 27132 68825 5879 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813969] [27148] 1012 27148 1099 25 1 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813971] [27149] 1012 27149 68825 5876 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813973] [27163] 1012 27163 1099 26 1 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813975] [27164] 1012 27164 4941 75 1 0 0 git
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813977] [27165] 1012 27165 68476 5523 1 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813979] [27171] 1012 27171 1099 26 0 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813981] [27172] 1012 27172 4984 61 1 0 0 git
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813984] [27173] 1012 27173 68476 5523 1 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813986] [27177] 1012 27177 1099 24 0 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813988] [27178] 1012 27178 10901 175 0 0 0 ssh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813990] [27179] 1012 27179 68579 5601 0 0 0 php
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813992] [27183] 1012 27183 1099 24 0 0 0 sh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813994] [27184] 1012 27184 10901 175 0 0 0 ssh
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.813996] Out of memory: Kill process 26620 (php) score 566 or sacrifice child
Jul 16 17:13:43 ip-10-61-174-46 kernel: [1521718.814006] Killed process 26620 (php) total-vm:2424180kB, anon-rss:2171272kB, file-rss:0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552063] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552068] Pid: 27179, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552070] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552072] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552083] [<ffffffff810e119f>] ? rcu_enter_nohz+0x4f/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552087] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552093] [<ffffffff81162156>] ? allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552096] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552100] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552103] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552109] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552113] [<ffffffff81057135>] ? reweight_entity+0x165/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552117] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552121] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552124] [<ffffffff8105746e>] ? entity_tick+0x11e/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552129] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552134] [<ffffffff811119fe>] ? perf_event_task_tick+0x1e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552138] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552142] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552145] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552149] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552152] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552155] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552159] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552162] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552166] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552169] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552173] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552176] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552180] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552185] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552189] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552191] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552196] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552199] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552201] <EOI> [<ffffffff8117c8c8>] ? prune_super+0x18/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552208] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552211] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552215] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552218] [<ffffffff81120133>] ? free_pcppages_bulk+0x2b3/0x3f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552221] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552225] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552229] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552233] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552236] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552240] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552244] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552247] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552251] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552254] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552257] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552261] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552265] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552268] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552272] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552275] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552278] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552280] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552282] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552284] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552286] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552288] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552290] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552294] active_anon:931167 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552295] active_file:21 inactive_file:0 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552296] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552297] free:2822 slab_reclaimable:2500 slab_unreclaimable:3369
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552298] mapped:1385 shmem:1409 pagetables:3944 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552301] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552310] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552314] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724668kB inactive_anon:5592kB active_file:84kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5540kB shmem:5636kB slab_reclaimable:10000kB slab_unreclaimable:13476kB kernel_stack:864kB pagetables:15776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552324] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552328] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552337] Node 0 DMA32: 300*4kB 24*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3024kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552347] 1434 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552348] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552350] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552352] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552353] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552355] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552358] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552361] node 0: slabs: 214, objs: 3424, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552368] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552370] Pid: 27179, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552372] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552373] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552378] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552382] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552385] [<ffffffff813ab2a7>] ? __xen_evtchn_do_upcall+0x2a7/0x310
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552388] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552392] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552395] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552398] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552401] [<ffffffff81057135>] ? reweight_entity+0x165/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552404] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552408] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552411] [<ffffffff8105746e>] ? entity_tick+0x11e/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552414] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552417] [<ffffffff811119fe>] ? perf_event_task_tick+0x1e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552420] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552424] [<ffffffff81077e38>] ? lock_timer_base.isra.29+0x38/0x70
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552427] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552430] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552432] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552435] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552438] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552441] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552445] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552448] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552451] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552454] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552457] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552460] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552463] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552466] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552469] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552472] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552475] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552477] <EOI> [<ffffffff8117c8c8>] ? prune_super+0x18/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552482] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552485] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552488] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552491] [<ffffffff81120133>] ? free_pcppages_bulk+0x2b3/0x3f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552494] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552498] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552501] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552504] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552507] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552510] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552513] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552516] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552519] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552522] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552525] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552528] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552531] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552534] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552537] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552540] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552542] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552543] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552545] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552547] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552548] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552550] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552552] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552556] active_anon:931167 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552556] active_file:21 inactive_file:0 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552557] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552558] free:2822 slab_reclaimable:2500 slab_unreclaimable:3369
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552559] mapped:1385 shmem:1409 pagetables:3944 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552561] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552569] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552572] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724668kB inactive_anon:5592kB active_file:84kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5540kB shmem:5636kB slab_reclaimable:10000kB slab_unreclaimable:13476kB kernel_stack:864kB pagetables:15776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552580] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552584] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552593] Node 0 DMA32: 300*4kB 24*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3024kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552602] 1434 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552604] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552605] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552607] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552608] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552610] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552612] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.552615] node 0: slabs: 214, objs: 3424, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556062] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556064] Pid: 27179, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556066] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556067] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556072] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556076] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556079] [<ffffffff813ab2a7>] ? __xen_evtchn_do_upcall+0x2a7/0x310
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556084] [<ffffffff81312a6b>] ? radix_tree_lookup+0xb/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556087] [<ffffffff810dbe5e>] ? irq_get_irq_data+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556090] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556093] [<ffffffff813acc4b>] ? xen_send_IPI_one+0x2b/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556096] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556099] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556103] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556106] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556109] [<ffffffff810573bf>] ? entity_tick+0x6f/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556112] [<ffffffff8106fc13>] ? raise_softirq+0x73/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556115] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556118] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556121] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556125] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556127] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556130] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556133] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556136] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556139] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556142] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556144] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556148] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556151] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556155] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556158] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556162] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556164] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556167] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556170] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556172] <EOI> [<ffffffff8117be01>] ? put_super+0x11/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556178] [<ffffffff8117bf02>] ? drop_super+0x22/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556181] [<ffffffff8117ca22>] ? prune_super+0x172/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556184] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556187] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556190] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556193] [<ffffffff81120133>] ? free_pcppages_bulk+0x2b3/0x3f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556196] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556199] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556202] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556206] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556208] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556211] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556215] [<ffffffff8152e3bc>] ? do_sock_read.isra.12+0xdc/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556218] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556220] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556223] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556226] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556229] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556232] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556235] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556238] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556241] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556243] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556245] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556247] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556249] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556250] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556252] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556254] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556258] active_anon:931167 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556259] active_file:21 inactive_file:0 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556260] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556260] free:2822 slab_reclaimable:2500 slab_unreclaimable:3369
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556261] mapped:1385 shmem:1409 pagetables:3944 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556263] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556271] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556275] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724668kB inactive_anon:5592kB active_file:84kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5540kB shmem:5636kB slab_reclaimable:10000kB slab_unreclaimable:13476kB kernel_stack:864kB pagetables:15776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556283] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556286] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556296] Node 0 DMA32: 300*4kB 24*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3024kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556306] 1434 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556308] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556309] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556311] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556312] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556314] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556316] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.556318] node 0: slabs: 214, objs: 3424, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752065] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752070] Pid: 27002, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752072] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752074] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752085] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752090] [<ffffffff81162156>] ? allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752094] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752097] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752100] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752106] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752110] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752115] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752118] [<ffffffff8105746e>] ? entity_tick+0x11e/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752123] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752127] [<ffffffff811119fe>] ? perf_event_task_tick+0x1e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752131] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752135] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752138] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752141] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752144] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752147] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752151] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752154] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752158] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752160] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752164] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752168] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752171] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752176] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752180] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752182] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752186] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752190] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752191] <EOI> [<ffffffff81090448>] ? up_read+0x18/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752198] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752201] [<ffffffff8117befa>] ? drop_super+0x1a/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752204] [<ffffffff8117ca22>] ? prune_super+0x172/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752208] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752211] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752214] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752217] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752220] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752224] [<ffffffff8115a9a3>] ? alloc_pages_vma+0xb3/0x190
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752227] [<ffffffff8113afbc>] ? do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752230] [<ffffffff8113ea41>] ? handle_pte_fault+0x1e1/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752233] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752236] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752240] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752243] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752246] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752249] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752253] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752255] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752256] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752258] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752260] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752262] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752264] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752266] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752270] active_anon:931167 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752270] active_file:21 inactive_file:0 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752271] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752272] free:2822 slab_reclaimable:2500 slab_unreclaimable:3369
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752273] mapped:1385 shmem:1409 pagetables:3944 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752275] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752283] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752287] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724668kB inactive_anon:5592kB active_file:84kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5540kB shmem:5636kB slab_reclaimable:10000kB slab_unreclaimable:13476kB kernel_stack:864kB pagetables:15776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752295] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752299] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752309] Node 0 DMA32: 301*4kB 24*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3028kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752318] 1434 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752320] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752322] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752323] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752324] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752326] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752329] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752331] node 0: slabs: 214, objs: 3424, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752338] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752340] Pid: 27002, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752342] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752343] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752348] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752351] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752355] [<ffffffff813ab2a7>] ? __xen_evtchn_do_upcall+0x2a7/0x310
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752358] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752361] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752364] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752368] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752371] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752374] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752378] [<ffffffff8105746e>] ? entity_tick+0x11e/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752381] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752384] [<ffffffff811119fe>] ? perf_event_task_tick+0x1e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752388] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752392] [<ffffffff81077e38>] ? lock_timer_base.isra.29+0x38/0x70
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752395] [<ffffffff8165e30e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752398] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752401] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752404] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752407] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752410] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752413] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752417] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752419] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752423] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752426] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752429] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752433] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752436] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752439] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752442] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752445] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752447] <EOI> [<ffffffff81090448>] ? up_read+0x18/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752453] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752456] [<ffffffff8117befa>] ? drop_super+0x1a/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752459] [<ffffffff8117ca22>] ? prune_super+0x172/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752463] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752466] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752469] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752473] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752476] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752480] [<ffffffff8115a9a3>] ? alloc_pages_vma+0xb3/0x190
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752483] [<ffffffff8113afbc>] ? do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752487] [<ffffffff8113ea41>] ? handle_pte_fault+0x1e1/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752490] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752493] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752496] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752499] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752502] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752506] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752509] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752511] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752512] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752514] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752516] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752518] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752520] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752522] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752526] active_anon:931167 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752527] active_file:21 inactive_file:0 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752528] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752529] free:2822 slab_reclaimable:2500 slab_unreclaimable:3369
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752530] mapped:1385 shmem:1409 pagetables:3944 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752532] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752541] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752544] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724668kB inactive_anon:5592kB active_file:84kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5540kB shmem:5636kB slab_reclaimable:10000kB slab_unreclaimable:13476kB kernel_stack:864kB pagetables:15776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752554] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752557] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752567] Node 0 DMA32: 301*4kB 24*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3028kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752576] 1434 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752577] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752579] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752581] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752582] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752584] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752586] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.752589] node 0: slabs: 214, objs: 3424, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756059] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756062] Pid: 26813, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756063] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756065] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756070] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756073] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756075] [<ffffffff813ab2a7>] ? __xen_evtchn_do_upcall+0x2a7/0x310
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756078] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756080] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756083] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756085] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756088] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756090] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756092] [<ffffffff8105746e>] ? entity_tick+0x11e/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756095] [<ffffffff8108e73c>] ? run_posix_cpu_timers+0x2c/0x270
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756097] [<ffffffff811119fe>] ? perf_event_task_tick+0x1e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756100] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756102] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756105] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756107] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756109] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756111] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756113] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756115] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756118] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756120] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756122] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756124] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756126] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756129] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756132] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756134] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756136] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756138] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756140] <EOI> [<ffffffff81012403>] ? xen_spin_lock+0x23/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756144] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756146] [<ffffffff8117c814>] ? grab_super_passive+0x24/0xc0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756149] [<ffffffff8117c8fc>] ? prune_super+0x4c/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756151] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756153] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756155] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756158] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756160] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756163] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756166] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756168] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756170] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756172] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756175] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756177] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756179] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756182] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756184] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756186] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756188] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756191] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756192] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756193] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756195] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756196] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756197] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756198] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756199] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756202] active_anon:931167 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756202] active_file:21 inactive_file:0 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756203] unevictable:0 dirty:1 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756204] free:2822 slab_reclaimable:2500 slab_unreclaimable:3369
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756204] mapped:1385 shmem:1409 pagetables:3944 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756206] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756212] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756214] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724668kB inactive_anon:5592kB active_file:84kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:4kB writeback:0kB mapped:5540kB shmem:5636kB slab_reclaimable:10000kB slab_unreclaimable:13476kB kernel_stack:864kB pagetables:15776kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756220] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756223] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756229] Node 0 DMA32: 301*4kB 24*8kB 0*16kB 1*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3028kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756236] 1434 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756237] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756238] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756239] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756240] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756241] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756243] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521725.756244] node 0: slabs: 214, objs: 3424, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416130] ntpd: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416134] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416135] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416137] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416146] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416149] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416153] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416156] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416160] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416164] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416169] [<ffffffff81135599>] ? __mod_zone_page_state+0x49/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416172] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416174] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416176] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416180] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416183] [<ffffffff813aadfe>] ? info_for_irq+0xe/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416186] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416188] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416191] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416193] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416196] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416198] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416200] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416203] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416205] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416207] [<ffffffff8158bb0e>] __tcp_ack_snd_check+0x5e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416210] [<ffffffff8159386f>] tcp_rcv_established+0x28f/0x700
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416212] [<ffffffff8159bf2f>] tcp_v4_do_rcv+0xff/0x1d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416215] [<ffffffff8159d921>] tcp_v4_rcv+0x691/0x7f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416218] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416220] [<ffffffff81579685>] ip_local_deliver_finish+0xe5/0x280
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416222] [<ffffffff815799e8>] ip_local_deliver+0x88/0x90
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416225] [<ffffffff8157933d>] ip_rcv_finish+0x10d/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416227] [<ffffffff81579c25>] ip_rcv+0x235/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416229] [<ffffffff81544bde>] __netif_receive_skb+0x4de/0x560
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416231] [<ffffffff815459d0>] netif_receive_skb+0x80/0x90
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416235] [<ffffffff81485c09>] ? xennet_get_responses+0x169/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416237] [<ffffffff81485813>] handle_incoming_queue+0x133/0x140
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416241] [<ffffffff8100aa70>] ? xen_vcpuop_set_next_event+0x40/0x70
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416243] [<ffffffff81486549>] xennet_poll+0x309/0x4d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416246] [<ffffffff8154633c>] net_rx_action+0x12c/0x280
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416249] [<ffffffff810d98e9>] ? handle_irq_event+0x59/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416251] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416256] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416259] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416261] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416263] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416266] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416267] <EOI> [<ffffffff81012403>] ? xen_spin_lock+0x23/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416272] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416275] [<ffffffff8117be0d>] ? put_super+0x1d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416277] [<ffffffff8117bf02>] ? drop_super+0x22/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416280] [<ffffffff8117ca22>] ? prune_super+0x172/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416282] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416284] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416287] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416289] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416292] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416294] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416297] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416300] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416302] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416304] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416307] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416309] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416311] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416314] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416316] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416319] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416321] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416323] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416324] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416326] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416327] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416328] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416330] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416331] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416334] active_anon:931190 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416334] active_file:0 inactive_file:18 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416335] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416336] free:2792 slab_reclaimable:2454 slab_unreclaimable:3384
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416336] mapped:1356 shmem:1409 pagetables:3961 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416338] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416344] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416347] Node 0 DMA32 free:2832kB min:7848kB low:9808kB high:11772kB active_anon:3724760kB inactive_anon:5592kB active_file:0kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:9816kB slab_unreclaimable:13536kB kernel_stack:872kB pagetables:15844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416353] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416356] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416362] Node 0 DMA32: 300*4kB 2*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2832kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416369] 1427 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416370] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416372] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416373] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416374] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416375] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416377] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416379] node 0: slabs: 215, objs: 3440, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416390] ntpd: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416392] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416393] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416394] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416399] [<ffffffff8164a33f>] ? slab_out_of_memory+0xfc/0x115
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416402] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416405] [<ffffffff81135599>] ? __mod_zone_page_state+0x49/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416407] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416410] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416412] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416415] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416417] [<ffffffff8153798b>] ? __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416419] [<ffffffff8112190e>] ? __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416422] [<ffffffff811646c1>] kmem_cache_alloc_node+0x61/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416424] [<ffffffff8153798b>] __alloc_skb+0x4b/0x240
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416426] [<ffffffff81597d4d>] tcp_send_ack+0x2d/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416428] [<ffffffff8158bb0e>] __tcp_ack_snd_check+0x5e/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416431] [<ffffffff8159386f>] tcp_rcv_established+0x28f/0x700
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416433] [<ffffffff8159bf2f>] tcp_v4_do_rcv+0xff/0x1d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416435] [<ffffffff8159d921>] tcp_v4_rcv+0x691/0x7f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416438] [<ffffffff81162190>] ? new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416440] [<ffffffff8164aec0>] ? __slab_alloc+0x246/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416443] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416445] [<ffffffff81579685>] ip_local_deliver_finish+0xe5/0x280
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416446] [<ffffffff815799e8>] ip_local_deliver+0x88/0x90
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416449] [<ffffffff8157933d>] ip_rcv_finish+0x10d/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416451] [<ffffffff81579c25>] ip_rcv+0x235/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416453] [<ffffffff81544bde>] __netif_receive_skb+0x4de/0x560
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416455] [<ffffffff815459d0>] netif_receive_skb+0x80/0x90
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416458] [<ffffffff81485c09>] ? xennet_get_responses+0x169/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416460] [<ffffffff81485813>] handle_incoming_queue+0x133/0x140
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416462] [<ffffffff8100aa70>] ? xen_vcpuop_set_next_event+0x40/0x70
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416465] [<ffffffff81486549>] xennet_poll+0x309/0x4d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416467] [<ffffffff8154633c>] net_rx_action+0x12c/0x280
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416469] [<ffffffff810d98e9>] ? handle_irq_event+0x59/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416471] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416474] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416476] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416478] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416480] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416483] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416484] <EOI> [<ffffffff81012403>] ? xen_spin_lock+0x23/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416488] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416490] [<ffffffff8117be0d>] ? put_super+0x1d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416493] [<ffffffff8117bf02>] ? drop_super+0x22/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416495] [<ffffffff8117ca22>] ? prune_super+0x172/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416497] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416499] [<ffffffff8165e7c0>] ? retint_restore_args+0x5/0x6
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416502] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416504] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416506] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416509] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416511] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416514] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416516] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416518] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416520] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416522] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416524] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416526] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416528] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416531] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416533] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416534] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416535] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416537] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416538] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416539] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416541] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416542] CPU 1: hi: 186, btch: 31 usd: 30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416544] active_anon:931190 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416545] active_file:0 inactive_file:18 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416546] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416546] free:2792 slab_reclaimable:2454 slab_unreclaimable:3384
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416547] mapped:1356 shmem:1409 pagetables:3961 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416548] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416554] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416557] Node 0 DMA32 free:2832kB min:7848kB low:9808kB high:11772kB active_anon:3724760kB inactive_anon:5592kB active_file:0kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:9816kB slab_unreclaimable:13536kB kernel_stack:872kB pagetables:15844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416563] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416565] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416571] Node 0 DMA32: 300*4kB 2*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2832kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416578] 1427 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416579] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416580] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416581] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416582] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416583] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416585] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.416586] node 0: slabs: 215, objs: 3440, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616073] php: page allocation failure: order:0, mode:0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616078] Pid: 26813, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616080] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616082] <IRQ> [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616092] [<ffffffff810e119f>] ? rcu_enter_nohz+0x4f/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616096] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616102] [<ffffffff81001200>] ? hypercall_page+0x200/0x1000
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616106] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616110] [<ffffffff81162156>] allocate_slab+0x196/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616113] [<ffffffff81162190>] new_slab+0x30/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616119] [<ffffffff8164addf>] __slab_alloc+0x165/0x269
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616123] [<ffffffff81537890>] ? skb_clone+0x40/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616127] [<ffffffff81165a62>] kmem_cache_alloc+0x112/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616130] [<ffffffff81537890>] skb_clone+0x40/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616133] [<ffffffff81540f0f>] dev_queue_xmit_nit+0xdf/0x1d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616137] [<ffffffff81546ec5>] dev_hard_start_xmit+0x65/0x520
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616142] [<ffffffff8104ecbc>] ? resched_task+0x2c/0x80
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616145] [<ffffffff81563d7e>] sch_direct_xmit+0xfe/0x1d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616149] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616152] [<ffffffff815474b7>] dev_queue_xmit+0x137/0x420
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616156] [<ffffffff8157e28b>] ip_finish_output+0x16b/0x2f0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616159] [<ffffffff8157ede8>] ip_output+0x98/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616162] [<ffffffff8157726f>] ? ipv4_dst_check+0x2f/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616165] [<ffffffff8157e4e9>] ip_local_out+0x29/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616168] [<ffffffff8157e630>] ip_queue_xmit+0x140/0x3e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616171] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616174] [<ffffffff8159640d>] tcp_transmit_skb+0x37d/0x5d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616177] [<ffffffff81597dac>] tcp_send_ack+0x8c/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616180] [<ffffffff81598672>] tcp_delack_timer+0x132/0x230
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616183] [<ffffffff8100a83b>] ? do_stolen_accounting+0xfb/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616186] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616190] [<ffffffff81076d06>] call_timer_fn+0x46/0x160
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616193] [<ffffffff81598540>] ? tcp_probe_timer+0x120/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616196] [<ffffffff81078652>] run_timer_softirq+0x132/0x2a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616200] [<ffffffff810d972a>] ? handle_irq_event_percpu+0xaa/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616203] [<ffffffff8106f6f8>] __do_softirq+0xa8/0x210
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616208] [<ffffffff81668bec>] call_softirq+0x1c/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616212] [<ffffffff810164b5>] do_softirq+0x65/0xa0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616215] [<ffffffff8106fade>] irq_exit+0x8e/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616219] [<ffffffff813ad095>] xen_evtchn_do_upcall+0x35/0x50
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616222] [<ffffffff81668c3e>] xen_do_hypervisor_callback+0x1e/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616224] <EOI> [<ffffffff810903dc>] ? down_read_trylock+0xc/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616232] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616236] [<ffffffff8117c845>] ? grab_super_passive+0x55/0xc0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616239] [<ffffffff8117c8fc>] ? prune_super+0x4c/0x1a0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616243] [<ffffffff8112a849>] ? shrink_slab+0xa9/0x300
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616246] [<ffffffff8112d188>] ? do_try_to_free_pages+0x268/0x2d0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616249] [<ffffffff81120380>] ? page_alloc_cpu_notify+0x60/0x60
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616252] [<ffffffff8112d4cb>] ? try_to_free_pages+0xab/0x170
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616255] [<ffffffff811217d5>] ? __alloc_pages_nodemask+0x585/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616259] [<ffffffff81159056>] ? alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616263] [<ffffffff81118907>] ? __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616265] [<ffffffff8111a8d2>] ? filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616269] [<ffffffff8113b2a2>] ? __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616272] [<ffffffff8113e95a>] ? handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616275] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616278] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616281] [<ffffffff8113fc39>] ? handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616285] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616288] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616291] [<ffffffff81661e74>] ? do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616294] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616297] [<ffffffff8165ea95>] ? page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616299] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616300] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616303] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616305] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616306] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616308] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616310] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616314] active_anon:931190 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616315] active_file:0 inactive_file:18 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616315] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616316] free:2822 slab_reclaimable:2454 slab_unreclaimable:3384
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616317] mapped:1356 shmem:1409 pagetables:3961 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616319] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616327] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616331] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724760kB inactive_anon:5592kB active_file:0kB inactive_file:72kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:9816kB slab_unreclaimable:13536kB kernel_stack:872kB pagetables:15844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616339] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616342] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616352] Node 0 DMA32: 300*4kB 17*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2952kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616361] 1427 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616363] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616365] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616366] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616367] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616369] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616372] cache: kmalloc-256, object size: 256, buffer size: 256, default order: 0, min order: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521726.616374] node 0: slabs: 215, objs: 3440, free: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808146] php cpuset=/ mems_allowed=0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808150] Pid: 26813, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808152] Call Trace:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808159] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808163] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808165] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808169] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808173] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808178] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808181] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808185] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808188] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808193] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808196] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808199] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808203] [<ffffffff8152e47d>] ? sock_aio_read+0x2d/0x40
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808206] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808210] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808217] [<ffffffff81179a3d>] ? vfs_read+0x10d/0x180
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808221] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808224] Mem-Info:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808225] Node 0 DMA per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808228] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808230] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808231] Node 0 DMA32 per-cpu:
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808233] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808235] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808239] active_anon:931190 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808239] active_file:11 inactive_file:7 isolated_file:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808240] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808241] free:2822 slab_reclaimable:2454 slab_unreclaimable:3389
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808242] mapped:1356 shmem:1409 pagetables:3961 bounce:0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808244] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808253] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808257] Node 0 DMA32 free:2952kB min:7848kB low:9808kB high:11772kB active_anon:3724760kB inactive_anon:5592kB active_file:44kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5636kB slab_reclaimable:9816kB slab_unreclaimable:13556kB kernel_stack:872kB pagetables:15844kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3514 all_unreclaimable? yes
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808265] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808269] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808278] Node 0 DMA32: 300*4kB 17*8kB 1*16kB 0*32kB 1*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2952kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808287] 1427 total pagecache pages
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808289] 0 pages in swap cache
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808291] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808292] Free swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.808293] Total swap = 0kB
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814167] 985072 pages RAM
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814169] 24998 pages reserved
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814170] 4377 pages shared
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814171] 953969 pages non-shared
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814173] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814180] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814182] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814185] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814187] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814189] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814192] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814194] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814196] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814200] [ 790] 101 790 63477 662 0 0 0 rsyslogd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814202] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814204] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814206] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814208] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814210] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814213] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814215] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814217] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814220] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814222] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814224] [ 6563] 0 6563 89584 2512 0 0 0 php5-fpm
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814226] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814229] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814231] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814233] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814235] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814237] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814239] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814241] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814244] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814246] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814248] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814250] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814252] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814254] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814256] [ 8228] 1012 8228 70863 6290 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814258] [ 8230] 1012 8230 69242 6236 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814260] [ 8232] 1012 8232 69859 6866 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814262] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814264] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814266] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814269] [26760] 1012 26760 356688 293373 1 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814271] [26813] 1012 26813 122894 53534 1 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814273] [26887] 1012 26887 294666 231398 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814275] [26934] 1012 26934 147470 83826 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814277] [27002] 1012 27002 267473 204132 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814279] [27070] 1012 27070 71450 8442 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814281] [27179] 1012 27179 71386 8435 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814283] [27254] 1012 27254 67532 4581 0 0 0 php
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814285] Out of memory: Kill process 26760 (php) score 306 or sacrifice child
Jul 16 17:13:54 ip-10-61-174-46 kernel: [1521729.814295] Killed process 26760 (php) total-vm:1426752kB, anon-rss:1173492kB, file-rss:0kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519720] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519726] php cpuset=/ mems_allowed=0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519729] Pid: 27002, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519731] Call Trace:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519739] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519742] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519746] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519750] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519754] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519759] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519762] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519766] [<ffffffff8100635d>] ? pte_mfn_to_pfn+0x8d/0x110
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519770] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519773] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519776] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519778] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519780] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519782] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519785] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519787] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519790] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519794] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519797] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519801] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519803] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519805] Mem-Info:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519806] Node 0 DMA per-cpu:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519808] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519809] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519811] Node 0 DMA32 per-cpu:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519812] CPU 0: hi: 186, btch: 31 usd: 31
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519814] CPU 1: hi: 186, btch: 31 usd: 19
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519816] active_anon:929577 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519817] active_file:0 inactive_file:30 isolated_file:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519817] unevictable:0 dirty:19 writeback:0 unstable:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519818] free:3820 slab_reclaimable:2617 slab_unreclaimable:3410
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519819] mapped:1394 shmem:1409 pagetables:4118 bounce:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519820] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519827] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519830] Node 0 DMA32 free:6944kB min:7848kB low:9808kB high:11772kB active_anon:3718308kB inactive_anon:5592kB active_file:0kB inactive_file:120kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:76kB writeback:0kB mapped:5576kB shmem:5636kB slab_reclaimable:10468kB slab_unreclaimable:13640kB kernel_stack:912kB pagetables:16472kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:229 all_unreclaimable? yes
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519836] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519839] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519846] Node 0 DMA32: 1284*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 6944kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519852] 1446 total pagecache pages
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519853] 0 pages in swap cache
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519855] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519856] Free swap = 0kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.519857] Total swap = 0kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525683] 985072 pages RAM
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525686] 24998 pages reserved
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525687] 4457 pages shared
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525688] 952903 pages non-shared
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525690] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525699] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525703] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525706] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525709] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525712] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525716] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525719] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525722] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525725] [ 790] 101 790 63477 719 0 0 0 rsyslogd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525728] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525731] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525734] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525737] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525740] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525742] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525745] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525748] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525752] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525882] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525886] [ 6563] 0 6563 89584 2496 1 0 0 php5-fpm
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525889] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525892] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525895] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525899] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525902] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525905] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525908] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525911] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525914] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525917] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525921] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525924] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525927] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525929] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525932] [ 8228] 1012 8228 70863 6289 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525935] [ 8230] 1012 8230 69242 6236 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525938] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525940] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525943] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525946] [ 9981] 108 9981 6792 76 1 0 0 pickup
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525949] [26813] 1012 26813 143938 78189 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525952] [26887] 1012 26887 336781 273478 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525954] [26934] 1012 26934 254224 190912 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525957] [27002] 1012 27002 327564 264300 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525959] [27070] 1012 27070 71450 8454 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525962] [27179] 1012 27179 120566 50557 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525965] [27254] 1012 27254 71386 8445 1 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525968] [27257] 1012 27257 1099 25 0 0 0 sh
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525970] [27258] 1012 27258 68825 5882 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.525973] [27282] 1012 27282 1099 26 0 0 0 sh
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.526029] [27283] 1012 27283 5216 67 1 0 0 git
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.526033] [27284] 1012 27284 68476 5525 0 0 0 php
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.526036] [27288] 1012 27288 1099 25 0 0 0 sh
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.526038] [27289] 1012 27289 10901 165 1 0 0 ssh
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.526042] Out of memory: Kill process 26887 (php) score 285 or sacrifice child
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.526141] Killed process 26887 (php) total-vm:1347124kB, anon-rss:1093892kB, file-rss:4kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628094] php: page allocation failure: order:0, mode:0x280da
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628099] Pid: 26887, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628101] Call Trace:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628110] [<ffffffff8111db9d>] warn_alloc_failed+0xfd/0x150
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628116] [<ffffffff8113656c>] ? wait_iff_congested+0xac/0x160
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628121] [<ffffffff8108b8c0>] ? add_wait_queue+0x60/0x60
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628124] [<ffffffff8112190e>] __alloc_pages_nodemask+0x6be/0x8e0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628129] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628132] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628136] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628140] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628143] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628146] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628151] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628156] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628160] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628163] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628165] Mem-Info:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628167] Node 0 DMA per-cpu:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628169] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628171] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628172] Node 0 DMA32 per-cpu:
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628175] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628176] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628180] active_anon:929577 inactive_anon:1398 isolated_anon:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628181] active_file:0 inactive_file:0 isolated_file:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628182] unevictable:0 dirty:19 writeback:0 unstable:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628183] free:3292 slab_reclaimable:2506 slab_unreclaimable:3560
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628184] mapped:1394 shmem:1409 pagetables:4118 bounce:0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628186] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628196] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628200] Node 0 DMA32 free:4832kB min:7848kB low:9808kB high:11772kB active_anon:3718308kB inactive_anon:5592kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:76kB writeback:0kB mapped:5576kB shmem:5636kB slab_reclaimable:10024kB slab_unreclaimable:14240kB kernel_stack:912kB pagetables:16472kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1 all_unreclaimable? no
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628208] lowmem_reserve[]: 0 0 0 0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628212] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628221] Node 0 DMA32: 840*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 4832kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628231] 1446 total pagecache pages
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628232] 0 pages in swap cache
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628234] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628237] Free swap = 0kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.628238] Total swap = 0kB
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.635651] 985072 pages RAM
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.635653] 24998 pages reserved
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.635654] 4384 pages shared
Jul 16 17:13:58 ip-10-61-174-46 kernel: [1521733.635656] 953486 pages non-shared
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812139] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812143] php cpuset=/ mems_allowed=0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812147] Pid: 27179, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812149] Call Trace:
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812157] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812160] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812163] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812166] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812170] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812174] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812178] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812182] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812185] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812188] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812192] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812195] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812199] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812202] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812206] [<ffffffff8105665d>] ? set_next_entity+0xad/0xd0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812209] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812212] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812216] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812218] Mem-Info:
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812220] Node 0 DMA per-cpu:
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812222] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812226] Node 0 DMA32 per-cpu:
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812228] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812229] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812233] active_anon:931611 inactive_anon:1398 isolated_anon:0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812234] active_file:0 inactive_file:0 isolated_file:0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812235] unevictable:0 dirty:0 writeback:17 unstable:0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812235] free:4026 slab_reclaimable:2456 slab_unreclaimable:2942
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812236] mapped:1360 shmem:1409 pagetables:4042 bounce:0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812239] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812247] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812251] Node 0 DMA32 free:7768kB min:7848kB low:9808kB high:11772kB active_anon:3726444kB inactive_anon:5592kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:68kB mapped:5440kB shmem:5636kB slab_reclaimable:9824kB slab_unreclaimable:11768kB kernel_stack:912kB pagetables:16168kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:86 all_unreclaimable? yes
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812259] lowmem_reserve[]: 0 0 0 0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812261] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812268] Node 0 DMA32: 189*4kB 566*8kB 24*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812275] 1452 total pagecache pages
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812276] 0 pages in swap cache
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812277] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812278] Free swap = 0kB
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.812279] Total swap = 0kB
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818491] 985072 pages RAM
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818493] 24998 pages reserved
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818495] 12728 pages shared
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818496] 944398 pages non-shared
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818498] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818506] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818509] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818512] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818516] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818519] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818522] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818525] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818528] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818533] [ 790] 101 790 63477 748 0 0 0 rsyslogd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818536] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818539] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818542] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818545] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818548] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818551] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818554] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818558] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818561] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818564] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818567] [ 6563] 0 6563 89584 2534 0 0 0 php5-fpm
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818570] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818573] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818576] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818579] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818582] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818585] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818588] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818591] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818594] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818597] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818600] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818603] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818606] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818609] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818612] [ 8228] 1012 8228 70863 6289 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818615] [ 8230] 1012 8230 69242 6237 1 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818618] [ 8232] 1012 8232 69859 6866 1 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818621] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818624] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818627] [ 9981] 108 9981 6792 76 0 0 0 pickup
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818631] [26813] 1012 26813 184872 112417 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818634] [26934] 1012 26934 313804 250548 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818636] [27002] 1012 27002 420882 357610 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818639] [27179] 1012 27179 226090 144608 1 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818642] [27254] 1012 27254 71450 8449 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818645] [27311] 1012 27311 71386 8428 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818648] [27315] 1012 27315 1099 25 0 0 0 sh
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818651] [27316] 1012 27316 68825 5872 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818654] [27328] 1012 27328 1099 26 1 0 0 sh
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818657] [27329] 1012 27329 1099 26 1 0 0 sh
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818660] [27330] 1012 27330 71450 8449 0 0 0 php
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818662] [27331] 1012 27331 2745 17 0 0 0 git
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818665] [27332] 1012 27332 547 1 0 0 0 git
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818668] Out of memory: Kill process 27002 (php) score 373 or sacrifice child
Jul 16 17:14:03 ip-10-61-174-46 kernel: [1521738.818680] Killed process 27002 (php) total-vm:1683528kB, anon-rss:1430440kB, file-rss:0kB
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824140] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824144] php cpuset=/ mems_allowed=0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824148] Pid: 8228, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824150] Call Trace:
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824158] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824161] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824164] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824168] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824172] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824177] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824179] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824184] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824188] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824191] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824194] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824198] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824201] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824205] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824209] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824214] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824217] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824221] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824225] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824227] Mem-Info:
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824228] Node 0 DMA per-cpu:
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824231] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824233] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824234] Node 0 DMA32 per-cpu:
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824237] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824239] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824242] active_anon:932058 inactive_anon:1398 isolated_anon:0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824243] active_file:9 inactive_file:14 isolated_file:0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824244] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824245] free:4038 slab_reclaimable:2395 slab_unreclaimable:2926
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824246] mapped:1379 shmem:1409 pagetables:3686 bounce:0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824248] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824255] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824258] Node 0 DMA32 free:7816kB min:7848kB low:9808kB high:11772kB active_anon:3728232kB inactive_anon:5592kB active_file:36kB inactive_file:56kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5636kB slab_reclaimable:9580kB slab_unreclaimable:11704kB kernel_stack:848kB pagetables:14744kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:312 all_unreclaimable? yes
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824265] lowmem_reserve[]: 0 0 0 0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824267] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824274] Node 0 DMA32: 820*4kB 317*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7928kB
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824281] 1463 total pagecache pages
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824282] 0 pages in swap cache
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824284] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824285] Free swap = 0kB
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.824285] Total swap = 0kB
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830181] 985072 pages RAM
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830183] 24998 pages reserved
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830184] 4392 pages shared
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830186] 952718 pages non-shared
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830187] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830197] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830200] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830204] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830207] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830210] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830213] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830217] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830220] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830224] [ 790] 101 790 63477 748 0 0 0 rsyslogd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830227] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830231] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830234] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830237] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830240] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830243] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830246] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830249] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830253] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830256] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830259] [ 6563] 0 6563 89584 2527 0 0 0 php5-fpm
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830262] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830265] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830268] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830271] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830274] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830277] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830280] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830284] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830287] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830290] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830293] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830296] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830299] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830302] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830305] [ 8228] 1012 8228 70863 6292 1 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830308] [ 8230] 1012 8230 69242 6236 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830312] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830315] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830318] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830321] [ 9981] 108 9981 6792 76 0 0 0 pickup
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830325] [26813] 1012 26813 319308 256022 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830328] [26934] 1012 26934 420498 357249 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830331] [27179] 1012 27179 326860 263585 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830334] [27338] 1012 27338 71450 8471 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830337] [27457] 1012 27457 65728 3279 0 0 0 php
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830340] Out of memory: Kill process 26934 (php) score 372 or sacrifice child
Jul 16 17:14:11 ip-10-61-174-46 kernel: [1521746.830352] Killed process 26934 (php) total-vm:1681992kB, anon-rss:1428984kB, file-rss:12kB
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680191] php invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680196] php cpuset=/ mems_allowed=0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680198] Pid: 27179, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680200] Call Trace:
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680208] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680210] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680212] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680215] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680219] [<ffffffff8115a9a3>] alloc_pages_vma+0xb3/0x190
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680222] [<ffffffff8113afbc>] do_anonymous_page.isra.38+0x7c/0x2f0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680225] [<ffffffff8113ea41>] handle_pte_fault+0x1e1/0x200
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680228] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680230] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680233] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680236] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680239] [<ffffffff8114413f>] ? do_brk+0x21f/0x320
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680241] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680244] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680247] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680248] Mem-Info:
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680250] Node 0 DMA per-cpu:
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680252] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680253] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680254] Node 0 DMA32 per-cpu:
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680256] CPU 0: hi: 186, btch: 31 usd: 1
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680257] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680260] active_anon:932268 inactive_anon:1398 isolated_anon:0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680260] active_file:31 inactive_file:0 isolated_file:0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680261] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680262] free:4048 slab_reclaimable:2409 slab_unreclaimable:2913
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680262] mapped:1354 shmem:1409 pagetables:3596 bounce:0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680264] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680271] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680274] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3729072kB inactive_anon:5592kB active_file:124kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5636kB slab_reclaimable:9636kB slab_unreclaimable:11652kB kernel_stack:840kB pagetables:14384kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32823 all_unreclaimable? yes
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680280] lowmem_reserve[]: 0 0 0 0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680283] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680289] Node 0 DMA32: 733*4kB 359*8kB 9*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8028kB
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680296] 1432 total pagecache pages
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680297] 0 pages in swap cache
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680299] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680300] Free swap = 0kB
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.680301] Total swap = 0kB
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685733] 985072 pages RAM
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685735] 24998 pages reserved
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685737] 4376 pages shared
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685738] 952704 pages non-shared
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685740] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685749] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685752] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685756] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685759] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685762] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685765] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685768] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685772] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685776] [ 790] 101 790 63477 748 0 0 0 rsyslogd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685779] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685782] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685785] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685788] [ 983] 108 983 6865 118 0 0 0 qmgr
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685791] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685795] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685798] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685801] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685804] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685807] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685810] [ 6563] 0 6563 89584 2511 0 0 0 php5-fpm
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685814] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685817] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685820] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685823] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685826] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685829] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685832] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685835] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685838] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685866] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685869] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685872] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685875] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685878] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685882] [ 8228] 1012 8228 70927 6298 1 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685885] [ 8230] 1012 8230 69242 6237 1 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685888] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685890] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685893] [ 8253] 1012 8253 68618 5636 0 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685896] [ 9981] 108 9981 6792 76 0 0 0 pickup
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685900] [26813] 1012 26813 518744 455519 0 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685903] [27179] 1012 27179 479638 416355 0 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685906] [27572] 1012 27572 71450 8464 1 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685909] [27634] 1012 27634 71450 8447 1 0 0 php
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685912] Out of memory: Kill process 26813 (php) score 475 or sacrifice child
Jul 16 17:14:21 ip-10-61-174-46 kernel: [1521756.685922] Killed process 26813 (php) total-vm:2074976kB, anon-rss:1822076kB, file-rss:0kB
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806545] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806550] ntpd cpuset=/ mems_allowed=0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806553] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806555] Call Trace:
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806563] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806567] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806570] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806573] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806577] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806582] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806584] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806588] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806592] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806596] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806598] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806601] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806605] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806610] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806612] Mem-Info:
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806614] Node 0 DMA per-cpu:
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806616] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806618] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806620] Node 0 DMA32 per-cpu:
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806622] CPU 0: hi: 186, btch: 31 usd: 0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806624] CPU 1: hi: 186, btch: 31 usd: 0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806628] active_anon:931933 inactive_anon:1398 isolated_anon:0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806629] active_file:0 inactive_file:37 isolated_file:0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806630] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806630] free:4030 slab_reclaimable:2380 slab_unreclaimable:2922
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806631] mapped:1377 shmem:1409 pagetables:3796 bounce:0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806634] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806642] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806646] Node 0 DMA32 free:7784kB min:7848kB low:9808kB high:11772kB active_anon:3727732kB inactive_anon:5592kB active_file:0kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5508kB shmem:5636kB slab_reclaimable:9520kB slab_unreclaimable:11688kB kernel_stack:928kB pagetables:15184kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806654] lowmem_reserve[]: 0 0 0 0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806658] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806668] Node 0 DMA32: 368*4kB 505*8kB 12*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7784kB
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806677] 1459 total pagecache pages
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806679] 0 pages in swap cache
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806681] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806682] Free swap = 0kB
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.806683] Total swap = 0kB
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812553] 985072 pages RAM
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812556] 24998 pages reserved
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812557] 4397 pages shared
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812558] 952737 pages non-shared
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812560] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812568] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812572] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812575] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812578] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812581] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812584] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812587] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812591] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812595] [ 790] 101 790 63477 748 0 0 0 rsyslogd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812598] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812601] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812604] [ 975] 0 975 6276 79 1 0 0 master
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812607] [ 983] 108 983 6850 125 0 0 0 qmgr
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812610] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812613] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812616] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812620] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812623] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812626] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812629] [ 6563] 0 6563 89584 2529 0 0 0 php5-fpm
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812632] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812635] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812638] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812641] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812644] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812647] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812650] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812654] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812657] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812660] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812663] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812666] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812669] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812672] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812675] [ 8228] 1012 8228 70927 6307 0 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812678] [ 8230] 1012 8230 69242 6237 1 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812681] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812684] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812687] [ 8253] 1012 8253 68618 5636 1 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812690] [ 9981] 108 9981 6792 76 0 0 0 pickup
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812693] [27179] 1012 27179 930737 867516 0 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812696] [27703] 108 27703 6818 73 0 0 0 cleanup
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812699] [27705] 108 27705 6795 71 0 0 0 trivial-rewrite
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812703] [27708] 108 27708 11107 159 1 0 0 smtp
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812706] [27709] 108 27709 11107 159 1 0 0 smtp
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812709] [27713] 108 27713 11107 158 1 0 0 smtp
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812712] [27717] 108 27717 11107 141 1 0 0 smtp
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812715] [28013] 1012 28013 1099 24 0 0 0 sh
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812718] [28014] 1012 28014 68825 5880 0 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812721] [28018] 1012 28018 71386 8427 0 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812724] [28028] 1012 28028 1099 26 0 0 0 sh
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812727] [28029] 1012 28029 5216 84 0 0 0 git
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812730] [28030] 1012 28030 68476 5523 1 0 0 php
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812733] [28034] 1012 28034 1099 25 0 0 0 sh
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812736] [28035] 1012 28035 10901 180 1 0 0 ssh
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812739] Out of memory: Kill process 27179 (php) score 905 or sacrifice child
Jul 16 17:14:37 ip-10-61-174-46 kernel: [1521772.812749] Killed process 27179 (php) total-vm:3722948kB, anon-rss:3470064kB, file-rss:0kB
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828563] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828567] php cpuset=/ mems_allowed=0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828569] Pid: 5666, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828571] Call Trace:
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828578] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828581] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828583] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828586] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828589] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828592] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828595] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828598] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828600] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828604] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828606] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828609] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828611] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828614] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828617] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828619] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828621] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828624] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828626] Mem-Info:
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828627] Node 0 DMA per-cpu:
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828629] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828630] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828631] Node 0 DMA32 per-cpu:
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828633] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828634] CPU 1: hi: 186, btch: 31 usd: 53
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828637] active_anon:931390 inactive_anon:1398 isolated_anon:0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828638] active_file:0 inactive_file:0 isolated_file:0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828638] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828639] free:4040 slab_reclaimable:2642 slab_unreclaimable:3657
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828639] mapped:1399 shmem:1409 pagetables:3417 bounce:0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828641] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828647] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828650] Node 0 DMA32 free:7824kB min:7848kB low:9808kB high:11772kB active_anon:3725560kB inactive_anon:5592kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5596kB shmem:5636kB slab_reclaimable:10568kB slab_unreclaimable:14628kB kernel_stack:824kB pagetables:13668kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:80 all_unreclaimable? yes
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828657] lowmem_reserve[]: 0 0 0 0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828659] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828666] Node 0 DMA32: 1444*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7824kB
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828672] 1421 total pagecache pages
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828674] 0 pages in swap cache
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828675] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828676] Free swap = 0kB
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.828677] Total swap = 0kB
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834160] 985072 pages RAM
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834162] 24998 pages reserved
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834163] 4406 pages shared
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834165] 952645 pages non-shared
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834166] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834175] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834179] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834182] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834185] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834189] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834192] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834195] [ 765] 0 765 12508 152 0 -17 -1000 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834199] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834203] [ 790] 101 790 63477 748 1 0 0 rsyslogd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834206] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834209] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834212] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834215] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834219] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834222] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834225] [ 1051] 0 1051 201625 844 1 0 0 collectd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834228] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834231] [ 1242] 107 1242 9445 163 1 0 0 ntpd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834234] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834238] [ 6563] 0 6563 89584 2496 0 0 0 php5-fpm
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834241] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834244] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834247] [10832] 0 10832 18359 200 0 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834250] [11050] 1002 11050 18359 201 0 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834253] [11051] 1002 11051 6542 1749 0 0 0 bash
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834256] [11156] 0 11156 18359 199 1 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834259] [11362] 1002 11362 18359 196 1 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834262] [11380] 1002 11380 6542 1750 1 0 0 bash
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834265] [25190] 0 25190 18359 200 1 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834268] [25434] 1002 25434 18741 602 1 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834271] [25450] 1002 25450 6542 1748 1 0 0 bash
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834275] [ 7907] 0 7907 18359 200 0 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834278] [ 8066] 1010 8066 18359 197 1 0 0 sshd
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834280] [ 8067] 1010 8067 6550 1759 0 0 0 bash
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834284] [ 8228] 1012 8228 71887 7312 1 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834287] [ 8230] 1012 8230 69178 6229 1 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834290] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834293] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834296] [ 8253] 1012 8253 68618 5667 1 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834299] [ 6785] 108 6785 6792 70 0 0 0 pickup
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834302] [ 5444] 1012 5444 945780 882556 1 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834305] [ 5666] 1012 5666 66815 4365 0 0 0 php
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834308] Out of memory: Kill process 5444 (php) score 921 or sacrifice child
Jul 16 17:37:33 ip-10-61-174-46 kernel: [1523148.834319] Killed process 5444 (php) total-vm:3783120kB, anon-rss:3530224kB, file-rss:0kB
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828737] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828741] php cpuset=/ mems_allowed=0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828744] Pid: 8922, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828746] Call Trace:
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828753] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828755] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828757] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828760] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828764] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828767] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828769] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828772] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828775] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828778] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828780] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828783] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828786] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828788] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828791] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828795] [<ffffffff81196e8e>] ? vfsmount_lock_local_unlock+0x1e/0x30
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828798] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828800] [<ffffffff81198d7f>] ? mntput+0x1f/0x30
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828803] [<ffffffff8117aca3>] ? __fput+0x143/0x200
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828805] [<ffffffff8117ad85>] ? fput+0x25/0x30
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828808] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828810] Mem-Info:
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828811] Node 0 DMA per-cpu:
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828813] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828814] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828815] Node 0 DMA32 per-cpu:
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828817] CPU 0: hi: 186, btch: 31 usd: 94
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828818] CPU 1: hi: 186, btch: 31 usd: 148
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828821] active_anon:931013 inactive_anon:1396 isolated_anon:0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828821] active_file:7 inactive_file:0 isolated_file:0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828822] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828823] free:4037 slab_reclaimable:2893 slab_unreclaimable:3238
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828823] mapped:1379 shmem:1407 pagetables:3742 bounce:0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828825] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828832] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828835] Node 0 DMA32 free:7812kB min:7848kB low:9808kB high:11772kB active_anon:3724052kB inactive_anon:5584kB active_file:28kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5516kB shmem:5628kB slab_reclaimable:11572kB slab_unreclaimable:12952kB kernel_stack:840kB pagetables:14968kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:395 all_unreclaimable? yes
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828841] lowmem_reserve[]: 0 0 0 0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828844] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828850] Node 0 DMA32: 1441*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7812kB
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828857] 1443 total pagecache pages
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828858] 0 pages in swap cache
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828859] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828861] Free swap = 0kB
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.828861] Total swap = 0kB
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834240] 985072 pages RAM
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834243] 24998 pages reserved
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834244] 4116 pages shared
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834245] 952778 pages non-shared
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834247] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834256] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834259] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834263] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834266] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834269] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834273] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834276] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834279] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834283] [ 790] 101 790 63525 799 0 0 0 rsyslogd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834286] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834289] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834293] [ 975] 0 975 6276 80 0 0 0 master
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834296] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834299] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834302] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834305] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834308] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834311] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834314] [15657] 0 15657 4777 60 0 0 0 cron
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834317] [ 6563] 0 6563 89584 2529 1 0 0 php5-fpm
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834321] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834324] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834327] [ 8228] 1012 8228 77519 12924 0 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834330] [ 8230] 1012 8230 69178 6242 0 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834333] [ 8232] 1012 8232 69859 6867 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834336] [ 8252] 1012 8252 70080 5537 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834339] [ 8253] 1012 8253 68618 5670 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834342] [25785] 0 25785 18359 199 0 0 0 sshd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834345] [26020] 1002 26020 18359 196 1 0 0 sshd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834348] [26035] 1002 26035 6544 1751 0 0 0 bash
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834351] [27466] 0 27466 18359 199 1 0 0 sshd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834354] [27468] 1002 27468 90390 27449 0 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834357] [27663] 1002 27663 18359 210 0 0 0 sshd
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834360] [27667] 1002 27667 6544 1752 0 0 0 bash
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834363] [28387] 1002 28387 6396 60 0 0 0 screen
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834366] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834370] [28389] 1002 28389 5559 764 0 0 0 bash
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834373] [28640] 1002 28640 89010 25695 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834376] [ 5112] 108 5112 6792 70 0 0 0 pickup
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834379] [ 8703] 1012 8703 861505 798298 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834382] [ 8911] 1012 8911 71817 8887 0 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834385] [ 8915] 1012 8915 72911 9950 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834388] [ 8921] 1012 8921 69964 6996 1 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834391] [ 8922] 1012 8922 70036 7052 0 0 0 php
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834394] Out of memory: Kill process 8703 (php) score 833 or sacrifice child
Jul 16 19:19:40 ip-10-61-174-46 kernel: [1529275.834405] Killed process 8703 (php) total-vm:3446020kB, anon-rss:3193192kB, file-rss:0kB
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813368] collectd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813371] collectd cpuset=/ mems_allowed=0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813374] Pid: 1073, comm: collectd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813376] Call Trace:
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813383] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813385] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813387] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813390] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813393] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813397] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813399] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813402] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813407] [<ffffffff8165e56e>] ? _raw_spin_lock_irqsave+0x2e/0x40
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813409] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813413] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813415] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813417] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813420] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813422] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813425] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813430] [<ffffffff8108fc34>] ? hrtimer_start_range_ns+0x14/0x20
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813433] [<ffffffff810a152c>] ? do_futex+0x7c/0x1b0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813435] [<ffffffff810a17a2>] ? sys_futex+0x142/0x1a0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813437] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813440] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813442] Mem-Info:
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813443] Node 0 DMA per-cpu:
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813445] CPU 0: hi: 0, btch: 1 usd: 0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813446] CPU 1: hi: 0, btch: 1 usd: 0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813447] Node 0 DMA32 per-cpu:
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813449] CPU 0: hi: 186, btch: 31 usd: 30
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813450] CPU 1: hi: 186, btch: 31 usd: 19
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813453] active_anon:931440 inactive_anon:1396 isolated_anon:0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813454] active_file:9 inactive_file:35 isolated_file:0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813454] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813455] free:4042 slab_reclaimable:2553 slab_unreclaimable:3225
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813456] mapped:1337 shmem:1407 pagetables:3749 bounce:0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813457] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813464] lowmem_reserve[]: 0 3772 3772 3772
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813467] Node 0 DMA32 free:7832kB min:7848kB low:9808kB high:11772kB active_anon:3725760kB inactive_anon:5584kB active_file:36kB inactive_file:140kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5348kB shmem:5628kB slab_reclaimable:10212kB slab_unreclaimable:12900kB kernel_stack:888kB pagetables:14996kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:263 all_unreclaimable? yes
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813474] lowmem_reserve[]: 0 0 0 0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813476] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813483] Node 0 DMA32: 767*4kB 280*8kB 32*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7868kB
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813490] 1440 total pagecache pages
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813491] 0 pages in swap cache
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813492] Swap cache stats: add 0, delete 0, find 0/0
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813493] Free swap = 0kB
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.813494] Total swap = 0kB
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820124] 985072 pages RAM
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820126] 24998 pages reserved
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820128] 4135 pages shared
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820129] 952931 pages non-shared
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820131] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820139] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820143] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820146] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820149] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820153] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820156] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820159] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820163] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820166] [ 790] 101 790 63525 799 0 0 0 rsyslogd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820170] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820173] [ 831] 0 831 4226 41 0 0 0 atd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820176] [ 975] 0 975 6276 79 0 0 0 master
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820179] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820182] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820185] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820188] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820191] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820194] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820197] [15657] 0 15657 4777 59 0 0 0 cron
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820200] [ 6563] 0 6563 89584 2512 0 0 0 php5-fpm
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820204] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820207] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820210] [ 8228] 1012 8228 77519 12941 1 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820213] [ 8230] 1012 8230 69178 6242 0 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820216] [ 8232] 1012 8232 69859 6866 0 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820219] [ 8252] 1012 8252 70080 5537 0 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820222] [ 8253] 1012 8253 68618 5670 1 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820225] [25785] 0 25785 18359 199 0 0 0 sshd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820228] [26020] 1002 26020 18359 203 1 0 0 sshd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820231] [26035] 1002 26035 6544 1751 0 0 0 bash
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820234] [27466] 0 27466 18359 199 1 0 0 sshd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820237] [27468] 1002 27468 90390 27449 0 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820240] [27663] 1002 27663 18359 210 0 0 0 sshd
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820243] [27667] 1002 27667 6544 1752 0 0 0 bash
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820246] [28387] 1002 28387 6396 60 0 0 0 screen
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820249] [28388] 1002 28388 6462 147 1 0 0 screen
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820252] [28389] 1002 28389 5559 764 0 0 0 bash
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820255] [28640] 1002 28640 89010 25695 0 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820258] [ 5112] 108 5112 6792 76 0 0 0 pickup
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820262] [ 8915] 1012 8915 876607 813343 1 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820264] [ 8937] 108 8937 6818 74 0 0 0 cleanup
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820268] [ 8938] 108 8938 6795 75 0 0 0 trivial-rewrite
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820271] [ 8939] 108 8939 11107 157 1 0 0 smtp
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820274] [ 8943] 108 8943 11107 156 0 0 0 smtp
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820277] [ 8947] 108 8947 11107 158 0 0 0 smtp
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820280] [ 8951] 108 8951 11107 157 0 0 0 smtp
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820283] [ 8957] 108 8957 11107 158 1 0 0 smtp
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820286] [ 8961] 108 8961 6792 70 1 0 0 scache
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820289] [ 9176] 1012 9176 71436 8517 0 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820292] [ 9203] 1012 9203 71651 8716 1 0 0 php
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820294] Out of memory: Kill process 8915 (php) score 848 or sacrifice child
Jul 16 19:20:13 ip-10-61-174-46 kernel: [1529308.820306] Killed process 8915 (php) total-vm:3506428kB, anon-rss:3253372kB, file-rss:0kB
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504168] sendmail invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504172] sendmail cpuset=/ mems_allowed=0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504174] Pid: 12277, comm: sendmail Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504176] Call Trace:
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504183] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504185] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504188] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504190] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504194] [<ffffffff8111804a>] ? unlock_page+0x2a/0x40
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504198] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504200] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504204] [<ffffffff812d8b0c>] ? apparmor_file_alloc_security+0x2c/0x60
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504206] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504208] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504212] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504215] [<ffffffff81177941>] ? fd_install+0x61/0x80
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504219] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504222] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504224] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504226] Mem-Info:
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504227] Node 0 DMA per-cpu:
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504229] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504231] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504232] Node 0 DMA32 per-cpu:
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504234] CPU 0: hi: 186, btch: 31 usd: 0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504235] CPU 1: hi: 186, btch: 31 usd: 0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504238] active_anon:931175 inactive_anon:1395 isolated_anon:0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504238] active_file:51 inactive_file:104 isolated_file:0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504239] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504240] free:4139 slab_reclaimable:2381 slab_unreclaimable:3028
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504240] mapped:1391 shmem:1406 pagetables:4066 bounce:0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504242] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504248] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504251] Node 0 DMA32 free:8220kB min:7848kB low:9808kB high:11772kB active_anon:3724700kB inactive_anon:5580kB active_file:204kB inactive_file:416kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5564kB shmem:5624kB slab_reclaimable:9524kB slab_unreclaimable:12112kB kernel_stack:992kB pagetables:16264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1664 all_unreclaimable? yes
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504258] lowmem_reserve[]: 0 0 0 0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504260] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504269] Node 0 DMA32: 1443*4kB 26*8kB 8*16kB 2*32kB 1*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 8284kB
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504277] 1574 total pagecache pages
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504278] 0 pages in swap cache
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504280] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504281] Free swap = 0kB
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.504281] Total swap = 0kB
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510270] 985072 pages RAM
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510272] 24998 pages reserved
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510274] 4033 pages shared
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510275] 953041 pages non-shared
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510277] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510286] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510289] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510292] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510295] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510299] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510302] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510305] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510308] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510312] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510315] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510318] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510321] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510324] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510327] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510330] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510333] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510336] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510340] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510343] [15657] 0 15657 4777 60 0 0 0 cron
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510346] [ 6563] 0 6563 89584 2501 0 0 0 php5-fpm
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510349] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510352] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510356] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510359] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510362] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510365] [10540] 1002 10540 18359 197 1 0 0 sshd
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510368] [10541] 1002 10541 6532 1740 0 0 0 bash
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510371] [10676] 1012 10676 70287 5756 1 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510374] [10678] 1012 10678 69178 6208 1 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510377] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510380] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510383] [10701] 1012 10701 68618 5624 1 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510386] [11362] 1012 11362 918154 854940 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510389] [11945] 108 11945 6792 75 0 0 0 pickup
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510392] [12074] 108 12074 6818 81 0 0 0 cleanup
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510395] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510398] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510402] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510405] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510408] [12115] 108 12115 6818 76 0 0 0 cleanup
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510411] [12116] 108 12116 11107 159 1 0 0 smtp
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510414] [12117] 108 12117 6792 75 1 0 0 scache
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510417] [12192] 1012 12192 1099 25 0 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510420] [12195] 1012 12195 68825 5890 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510423] [12198] 1012 12198 1099 24 0 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510426] [12199] 1012 12199 68825 5882 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510429] [12203] 1012 12203 1099 24 1 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510432] [12204] 1012 12204 68825 5874 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510435] [12238] 1012 12238 1099 25 1 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510438] [12239] 1012 12239 69594 6645 1 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510441] [12240] 1012 12240 67816 4899 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510443] [12258] 1012 12258 1099 26 0 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510446] [12260] 1012 12260 1099 26 0 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510449] [12261] 1012 12261 1099 25 0 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510452] [12263] 1012 12263 4651 57 0 0 0 git
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510455] [12265] 1012 12265 4984 63 0 0 0 git
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510458] [12266] 1012 12266 4651 58 0 0 0 git
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510461] [12267] 1012 12267 64811 2037 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510464] [12268] 1012 12268 64727 2087 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510467] [12269] 1012 12269 64811 2060 0 0 0 php
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510470] [12276] 1012 12276 1099 24 1 0 0 sh
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510472] [12277] 1012 12277 6272 66 1 0 0 sendmail
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510475] Out of memory: Kill process 11362 (php) score 892 or sacrifice child
Jul 17 00:05:35 ip-10-61-174-46 kernel: [1546431.510487] Killed process 11362 (php) total-vm:3672616kB, anon-rss:3419760kB, file-rss:0kB
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780159] php invoked oom-killer: gfp_mask=0xd0, order=1, oom_adj=0, oom_score_adj=0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780163] php cpuset=/ mems_allowed=0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780166] Pid: 13432, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780168] Call Trace:
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780174] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780177] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780179] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780182] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780186] [<ffffffff8111804a>] ? unlock_page+0x2a/0x40
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780189] [<ffffffff81065861>] dup_task_struct+0x71/0x1d0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780191] [<ffffffff810661ce>] copy_process.part.18+0x6e/0xea0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780195] [<ffffffff811648ac>] ? kmem_cache_alloc_trace+0xfc/0x120
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780198] [<ffffffff81067077>] copy_process+0x77/0x80
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780200] [<ffffffff810671ca>] do_fork+0xfa/0x340
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780204] [<ffffffff8165e2de>] ? _raw_spin_lock+0xe/0x20
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780206] [<ffffffff81177941>] ? fd_install+0x61/0x80
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780209] [<ffffffff81183c33>] ? do_pipe_flags+0xc3/0x120
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780213] [<ffffffff8101da08>] sys_clone+0x28/0x30
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780216] [<ffffffff81666ca3>] stub_clone+0x13/0x20
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780218] [<ffffffff81666982>] ? system_call_fastpath+0x16/0x1b
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780220] Mem-Info:
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780221] Node 0 DMA per-cpu:
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780223] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780225] Node 0 DMA32 per-cpu:
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780227] CPU 0: hi: 186, btch: 31 usd: 39
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780228] CPU 1: hi: 186, btch: 31 usd: 75
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780231] active_anon:930096 inactive_anon:1395 isolated_anon:0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780232] active_file:12 inactive_file:29 isolated_file:24
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780233] unevictable:0 dirty:4 writeback:0 unstable:0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780233] free:4034 slab_reclaimable:2800 slab_unreclaimable:3274
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780234] mapped:1400 shmem:1406 pagetables:4568 bounce:0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780236] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780242] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780245] Node 0 DMA32 free:7800kB min:7848kB low:9808kB high:11772kB active_anon:3720384kB inactive_anon:5580kB active_file:48kB inactive_file:116kB unevictable:0kB isolated(anon):0kB isolated(file):96kB present:3862656kB mlocked:0kB dirty:16kB writeback:0kB mapped:5600kB shmem:5624kB slab_reclaimable:11200kB slab_unreclaimable:13096kB kernel_stack:1112kB pagetables:18272kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:170 all_unreclaimable? yes
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780252] lowmem_reserve[]: 0 0 0 0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780254] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780261] Node 0 DMA32: 1378*4kB 30*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7800kB
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780268] 1435 total pagecache pages
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780269] 0 pages in swap cache
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780270] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780272] Free swap = 0kB
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.780272] Total swap = 0kB
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786431] 985072 pages RAM
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786433] 24998 pages reserved
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786434] 3952 pages shared
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786435] 953050 pages non-shared
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786437] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786445] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786449] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786452] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786455] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786457] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786460] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786463] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786466] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786471] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786474] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786477] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786480] [ 975] 0 975 6276 80 0 0 0 master
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786483] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786486] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786488] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786491] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786494] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786497] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786500] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786503] [ 6563] 0 6563 89584 2532 0 0 0 php5-fpm
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786506] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786509] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786512] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786515] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786519] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786522] [10540] 1002 10540 18359 197 1 0 0 sshd
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786525] [10541] 1002 10541 6532 1740 0 0 0 bash
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786528] [10676] 1012 10676 70351 5770 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786531] [10678] 1012 10678 69178 6217 1 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786534] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786537] [10700] 1012 10700 70080 5538 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786540] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786543] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786546] [12077] 108 12077 11107 161 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786549] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786552] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786555] [12116] 108 12116 11107 160 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786558] [12117] 108 12117 6825 75 0 0 0 scache
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786560] [12290] 108 12290 11107 161 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786563] [12312] 108 12312 11108 159 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786565] [12556] 108 12556 11108 159 1 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786568] [12561] 108 12561 11108 160 1 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786570] [12566] 108 12566 11108 160 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786573] [12579] 108 12579 11108 160 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786575] [12603] 108 12603 11108 158 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786578] [12607] 108 12607 11108 162 1 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786580] [12743] 108 12743 11108 160 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786583] [12778] 108 12778 11108 159 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786585] [12782] 108 12782 11108 159 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786588] [12805] 1012 12805 752301 689062 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786591] [12810] 108 12810 11108 160 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786593] [12816] 108 12816 11108 160 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786596] [12821] 108 12821 11108 162 0 0 0 smtp
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786598] [13176] 1012 13176 69658 6680 1 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786601] [13192] 108 13192 6792 69 0 0 0 pickup
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786603] [13196] 108 13196 6818 73 0 0 0 cleanup
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786606] [13294] 1012 13294 225067 149568 1 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786608] [13319] 108 13319 6818 74 0 0 0 cleanup
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786625] [13431] 1012 13431 1099 24 0 0 0 sh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786629] [13432] 1012 13432 68825 5891 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786631] [13433] 1012 13433 1099 25 1 0 0 sh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786634] [13434] 1012 13434 68825 5895 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786647] [13465] 1012 13465 1099 25 0 0 0 sh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786650] [13466] 1012 13466 68825 5874 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786652] [13488] 1012 13488 69595 6657 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786655] [13498] 1012 13498 1099 25 0 0 0 sh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786664] [13499] 1012 13499 5099 94 0 0 0 git
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786667] [13500] 1012 13500 68476 5523 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786670] [13502] 1012 13502 69448 6485 0 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786672] [13506] 1012 13506 68195 5206 1 0 0 php
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786675] [13511] 1012 13511 1099 25 1 0 0 sh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786678] [13512] 1012 13512 10901 180 1 0 0 ssh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786681] [13541] 1012 13541 19591 84 0 0 0 git
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786684] [13546] 1012 13546 1099 26 1 0 0 sh
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786687] Out of memory: Kill process 12805 (php) score 719 or sacrifice child
Jul 17 00:06:08 ip-10-61-174-46 kernel: [1546463.786699] Killed process 12805 (php) total-vm:3009204kB, anon-rss:2756248kB, file-rss:0kB
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894589] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894593] php cpuset=/ mems_allowed=0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894596] Pid: 10676, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894598] Call Trace:
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894605] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894608] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894610] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894613] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894616] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894619] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894622] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894625] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894628] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894630] [<ffffffff8118c650>] ? __pollwait+0xf0/0xf0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894633] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894636] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894639] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894641] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894645] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894648] [<ffffffff8109585d>] ? ktime_get_ts+0xad/0xe0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894651] [<ffffffff8118c3bd>] ? poll_select_copy_remaining+0xed/0x140
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894653] [<ffffffff8118d69d>] ? sys_select+0xcd/0x100
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894656] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894658] Mem-Info:
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894660] Node 0 DMA per-cpu:
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894662] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894663] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894664] Node 0 DMA32 per-cpu:
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894666] CPU 0: hi: 186, btch: 31 usd: 66
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894667] CPU 1: hi: 186, btch: 31 usd: 43
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894670] active_anon:930272 inactive_anon:1395 isolated_anon:0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894670] active_file:37 inactive_file:3 isolated_file:16
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894671] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894672] free:4018 slab_reclaimable:2662 slab_unreclaimable:3261
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894672] mapped:1370 shmem:1406 pagetables:4566 bounce:0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894674] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894680] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894683] Node 0 DMA32 free:7736kB min:7848kB low:9808kB high:11772kB active_anon:3721088kB inactive_anon:5580kB active_file:148kB inactive_file:12kB unevictable:0kB isolated(anon):0kB isolated(file):64kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5480kB shmem:5624kB slab_reclaimable:10648kB slab_unreclaimable:13044kB kernel_stack:1088kB pagetables:18264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:335 all_unreclaimable? yes
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894690] lowmem_reserve[]: 0 0 0 0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894692] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894699] Node 0 DMA32: 609*4kB 105*8kB 153*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7804kB
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894706] 1449 total pagecache pages
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894707] 0 pages in swap cache
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894708] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894709] Free swap = 0kB
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.894710] Total swap = 0kB
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900178] 985072 pages RAM
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900180] 24998 pages reserved
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900182] 3932 pages shared
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900183] 953065 pages non-shared
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900185] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900193] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900197] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900200] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900203] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900207] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900210] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900213] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900217] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900221] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900224] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900227] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900230] [ 975] 0 975 6276 79 1 0 0 master
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900233] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900237] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900240] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900243] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900246] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900249] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900252] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900255] [ 6563] 0 6563 89584 2516 1 0 0 php5-fpm
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900259] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900262] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900265] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900268] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900272] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900275] [10540] 1002 10540 18359 197 1 0 0 sshd
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900278] [10541] 1002 10541 6532 1740 0 0 0 bash
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900281] [10676] 1012 10676 70351 5808 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900284] [10678] 1012 10678 69178 6217 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900287] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900290] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900293] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900296] [12075] 108 12075 6795 75 1 0 0 trivial-rewrite
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900299] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900303] [12107] 108 12107 11107 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900306] [12108] 108 12108 11107 161 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900309] [12116] 108 12116 11107 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900312] [12117] 108 12117 6819 93 1 0 0 scache
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900315] [12290] 108 12290 11107 161 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900318] [12312] 108 12312 11108 159 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900321] [12556] 108 12556 11108 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900324] [12561] 108 12561 11108 161 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900327] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900330] [12579] 108 12579 11108 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900333] [12603] 108 12603 11108 158 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900336] [12607] 108 12607 11108 162 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900339] [12743] 108 12743 11108 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900342] [12778] 108 12778 11108 159 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900345] [12782] 108 12782 11108 159 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900348] [12810] 108 12810 11108 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900352] [12816] 108 12816 11108 160 1 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900355] [12821] 108 12821 11108 161 0 0 0 smtp
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900358] [13294] 1012 13294 797360 734096 0 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900361] [14032] 108 14032 6818 82 1 0 0 cleanup
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900364] [14040] 1012 14040 69654 6663 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900367] [14053] 108 14053 6792 70 1 0 0 pickup
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900370] [14114] 1012 14114 177298 106767 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900373] [14135] 1012 14135 69594 6650 0 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900376] [14192] 1012 14192 1099 25 0 0 0 sh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900379] [14193] 1012 14193 68825 5875 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900382] [14201] 1012 14201 1099 24 0 0 0 sh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900385] [14202] 1012 14202 68825 5872 0 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900388] [14214] 1012 14214 1099 25 0 0 0 sh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900391] [14215] 1012 14215 4651 57 1 0 0 git
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900394] [14216] 1012 14216 68476 5523 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900397] [14222] 1012 14222 1099 26 0 0 0 sh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900400] [14223] 1012 14223 5065 69 0 0 0 git
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900403] [14224] 1012 14224 68399 5423 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900406] [14226] 1012 14226 68579 5606 0 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900409] [14228] 1012 14228 67006 4554 1 0 0 php
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900412] [14239] 1012 14239 1099 25 1 0 0 sh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900415] [14240] 1012 14240 10901 175 1 0 0 ssh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900418] [14241] 1012 14241 1099 25 1 0 0 sh
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900421] [14243] 1012 14243 6272 66 0 0 0 sendmail
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900424] [14246] 1012 14246 4673 34 1 0 0 postdrop
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900427] Out of memory: Kill process 13294 (php) score 766 or sacrifice child
Jul 17 00:06:28 ip-10-61-174-46 kernel: [1546483.900439] Killed process 13294 (php) total-vm:3189440kB, anon-rss:2936384kB, file-rss:0kB
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.115999] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116002] ntpd cpuset=/ mems_allowed=0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116005] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116007] Call Trace:
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116014] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116017] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116019] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116022] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116025] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116028] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116031] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116033] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116036] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116039] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116042] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116044] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116048] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116053] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116068] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116070] Mem-Info:
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116072] Node 0 DMA per-cpu:
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116074] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116076] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116078] Node 0 DMA32 per-cpu:
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116080] CPU 0: hi: 186, btch: 31 usd: 112
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116082] CPU 1: hi: 186, btch: 31 usd: 74
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116086] active_anon:930478 inactive_anon:1395 isolated_anon:0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116087] active_file:0 inactive_file:5 isolated_file:19
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116088] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116089] free:4041 slab_reclaimable:2648 slab_unreclaimable:3232
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116090] mapped:1365 shmem:1406 pagetables:4316 bounce:0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116092] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116100] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116104] Node 0 DMA32 free:7828kB min:7848kB low:9808kB high:11772kB active_anon:3721912kB inactive_anon:5580kB active_file:0kB inactive_file:20kB unevictable:0kB isolated(anon):0kB isolated(file):76kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5460kB shmem:5624kB slab_reclaimable:10592kB slab_unreclaimable:12928kB kernel_stack:1048kB pagetables:17264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:171 all_unreclaimable? yes
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116112] lowmem_reserve[]: 0 0 0 0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116116] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116126] Node 0 DMA32: 667*4kB 258*8kB 66*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7836kB
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116136] 1444 total pagecache pages
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116138] 0 pages in swap cache
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116140] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116141] Free swap = 0kB
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.116142] Total swap = 0kB
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121625] 985072 pages RAM
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121627] 24998 pages reserved
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121629] 3939 pages shared
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121630] 952978 pages non-shared
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121632] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121640] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121643] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121647] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121649] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121651] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121654] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121656] [ 765] 0 765 12508 153 1 -17 -1000 sshd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121659] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121662] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121664] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121666] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121669] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121671] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121673] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121675] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121678] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121680] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121682] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121684] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121686] [ 6563] 0 6563 89584 2500 0 0 0 php5-fpm
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121689] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121691] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121693] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121696] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121698] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121700] [10540] 1002 10540 18359 201 1 0 0 sshd
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121702] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121705] [10676] 1012 10676 70351 5829 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121707] [10678] 1012 10678 69178 6217 0 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121709] [10680] 1012 10680 71906 8988 0 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121711] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121713] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121715] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121717] [12077] 108 12077 11107 161 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121720] [12107] 108 12107 11107 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121722] [12108] 108 12108 11107 161 0 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121725] [12116] 108 12116 11107 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121727] [12117] 108 12117 6819 93 0 0 0 scache
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121729] [12290] 108 12290 11107 161 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121731] [12312] 108 12312 11108 158 0 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121733] [12556] 108 12556 11108 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121735] [12561] 108 12561 11108 161 0 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121737] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121739] [12579] 108 12579 11108 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121741] [12603] 108 12603 11108 159 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121743] [12607] 108 12607 11108 162 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121745] [12743] 108 12743 11108 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121747] [12778] 108 12778 11108 159 0 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121750] [12782] 108 12782 11108 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121752] [12810] 108 12810 11108 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121754] [12816] 108 12816 11108 160 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121756] [12821] 108 12821 11108 162 1 0 0 smtp
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121758] [14114] 1012 14114 909874 846613 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121760] [14454] 1002 14454 3115 44 1 0 0 less
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121762] [14560] 108 14560 6818 73 1 0 0 cleanup
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121764] [14594] 108 14594 6818 74 0 0 0 cleanup
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121767] [15006] 108 15006 6792 69 0 0 0 pickup
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121769] [15125] 1012 15125 71914 8980 0 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121771] [15140] 1012 15140 71913 8975 0 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121773] [15207] 1012 15207 1099 24 0 0 0 sh
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121775] [15208] 1012 15208 68825 5874 0 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121777] [15209] 1012 15209 1099 24 0 0 0 sh
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121779] [15212] 1012 15212 68825 5876 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121781] [15264] 1012 15264 1099 26 1 0 0 sh
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121783] [15267] 1012 15267 1099 25 1 0 0 sh
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121785] [15268] 1012 15268 5218 68 1 0 0 git
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121787] [15269] 1012 15269 4984 62 1 0 0 git
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121789] [15270] 1012 15270 66499 4050 0 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121791] [15271] 1012 15271 66499 4050 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121793] [15283] 1012 15283 65472 2971 1 0 0 php
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121796] Out of memory: Kill process 14114 (php) score 883 or sacrifice child
Jul 17 00:06:55 ip-10-61-174-46 kernel: [1546511.121805] Killed process 14114 (php) total-vm:3639496kB, anon-rss:3386452kB, file-rss:0kB
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209185] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209190] php cpuset=/ mems_allowed=0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209194] Pid: 15858, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209196] Call Trace:
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209204] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209208] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209211] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209215] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209221] [<ffffffff8165ed0a>] ? error_exit+0x2a/0x60
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209225] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209230] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209232] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209236] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209239] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209243] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209246] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209250] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209253] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209256] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209260] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209263] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209265] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209269] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209271] Mem-Info:
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209273] Node 0 DMA per-cpu:
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209276] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209278] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209279] Node 0 DMA32 per-cpu:
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209281] CPU 0: hi: 186, btch: 31 usd: 36
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209283] CPU 1: hi: 186, btch: 31 usd: 0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209287] active_anon:930395 inactive_anon:1396 isolated_anon:0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209288] active_file:8 inactive_file:101 isolated_file:0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209289] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209290] free:4019 slab_reclaimable:2733 slab_unreclaimable:3322
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209291] mapped:1387 shmem:1407 pagetables:4388 bounce:0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209293] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209301] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209305] Node 0 DMA32 free:7740kB min:7848kB low:9808kB high:11772kB active_anon:3721580kB inactive_anon:5584kB active_file:32kB inactive_file:404kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5548kB shmem:5628kB slab_reclaimable:10932kB slab_unreclaimable:13288kB kernel_stack:1056kB pagetables:17552kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:22 all_unreclaimable? no
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209314] lowmem_reserve[]: 0 0 0 0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209318] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209327] Node 0 DMA32: 1035*4kB 194*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7756kB
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209337] 1469 total pagecache pages
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209339] 0 pages in swap cache
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209343] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209345] Free swap = 0kB
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.209346] Total swap = 0kB
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216702] 985072 pages RAM
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216704] 24998 pages reserved
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216705] 4098 pages shared
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216706] 952945 pages non-shared
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216708] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216717] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216720] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216723] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216726] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216729] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216732] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216735] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216738] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216741] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216744] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216747] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216750] [ 975] 0 975 6276 80 0 0 0 master
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216753] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216756] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216760] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216762] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216765] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216768] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216771] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216774] [ 6563] 0 6563 89584 2526 0 0 0 php5-fpm
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216777] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216780] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216783] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216786] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216789] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216792] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216795] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216797] [10676] 1012 10676 70480 5898 0 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216800] [10678] 1012 10678 69178 6223 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216802] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216805] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216808] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216811] [12075] 108 12075 6795 75 1 0 0 trivial-rewrite
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216814] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216816] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216819] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216822] [12116] 108 12116 11107 160 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216825] [12117] 108 12117 6819 93 0 0 0 scache
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216828] [12290] 108 12290 11107 160 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216830] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216833] [12556] 108 12556 11108 160 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216836] [12561] 108 12561 11108 161 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216839] [12566] 108 12566 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216842] [12579] 108 12579 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216844] [12603] 108 12603 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216847] [12607] 108 12607 11108 161 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216850] [12743] 108 12743 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216853] [12778] 108 12778 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216856] [12782] 108 12782 11108 160 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216859] [12810] 108 12810 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216861] [12816] 108 12816 11108 159 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216864] [12821] 108 12821 11108 161 1 0 0 smtp
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216867] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216870] [14594] 108 14594 6818 82 0 0 0 cleanup
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216873] [15006] 108 15006 6792 75 0 0 0 pickup
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216875] [15283] 1012 15283 680104 616901 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216878] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216881] [15478] 1012 15478 78867 15900 0 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216883] [15505] 1012 15505 273262 210066 0 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216886] [15678] 1002 15678 18359 197 1 0 0 sshd
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216888] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216891] [15858] 1012 15858 79856 16885 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216893] [15904] 1012 15904 75792 12838 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216896] [15931] 1012 15931 1099 25 1 0 0 sh
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216898] [15932] 1012 15932 66680 4248 1 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216901] [15933] 1012 15933 1099 25 0 0 0 sh
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216904] [15936] 1012 15936 66678 4231 0 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216907] [15942] 1012 15942 1099 25 1 0 0 sh
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216909] [15943] 1012 15943 66678 4233 0 0 0 php
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216912] [15948] 108 15948 6818 73 1 0 0 cleanup
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216914] [15966] 1002 15966 3115 45 0 0 0 less
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216917] Out of memory: Kill process 15283 (php) score 643 or sacrifice child
Jul 17 00:07:22 ip-10-61-174-46 kernel: [1546538.216927] Killed process 15283 (php) total-vm:2720416kB, anon-rss:2467604kB, file-rss:0kB
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808167] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808172] php cpuset=/ mems_allowed=0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808175] Pid: 16210, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808177] Call Trace:
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808184] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808188] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808191] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808194] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808200] [<ffffffff8121a4d0>] ? noalloc_get_block_write+0x30/0x30
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808204] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808209] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808212] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808215] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808218] [<ffffffff8113ca82>] ? do_wp_page+0x382/0x740
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808222] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808226] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808229] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808233] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808236] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808239] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808244] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808248] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808251] [<ffffffff8100a8c0>] ? xen_clocksource_read+0x20/0x30
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808254] [<ffffffff8100a9a9>] ? xen_clocksource_get_cycles+0x9/0x10
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808259] [<ffffffff81095b27>] ? getnstimeofday+0x57/0xe0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808263] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808265] Mem-Info:
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808266] Node 0 DMA per-cpu:
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808269] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808271] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808272] Node 0 DMA32 per-cpu:
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808274] CPU 0: hi: 186, btch: 31 usd: 0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808276] CPU 1: hi: 186, btch: 31 usd: 0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808280] active_anon:930498 inactive_anon:1396 isolated_anon:0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808281] active_file:13 inactive_file:19 isolated_file:0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808281] unevictable:0 dirty:0 writeback:20 unstable:0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808282] free:2836 slab_reclaimable:2571 slab_unreclaimable:3655
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808283] mapped:1362 shmem:1407 pagetables:4221 bounce:0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808285] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808294] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808298] Node 0 DMA32 free:3008kB min:7848kB low:9808kB high:11772kB active_anon:3721992kB inactive_anon:5584kB active_file:52kB inactive_file:76kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:80kB mapped:5448kB shmem:5628kB slab_reclaimable:10284kB slab_unreclaimable:14620kB kernel_stack:1064kB pagetables:16884kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1172 all_unreclaimable? yes
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808306] lowmem_reserve[]: 0 0 0 0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808310] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808320] Node 0 DMA32: 305*4kB 78*8kB 1*16kB 0*32kB 0*64kB 1*128kB 0*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 3012kB
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808329] 1450 total pagecache pages
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808331] 0 pages in swap cache
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808332] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808334] Free swap = 0kB
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.808335] Total swap = 0kB
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813853] 985072 pages RAM
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813855] 24998 pages reserved
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813857] 4096 pages shared
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813858] 954213 pages non-shared
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813860] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813868] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813872] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813875] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813878] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813881] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813904] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813908] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813911] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813915] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813918] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813921] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813925] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813927] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813931] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813934] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813937] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813940] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813943] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813946] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813950] [ 6563] 0 6563 89584 2497 0 0 0 php5-fpm
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813953] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813956] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813959] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813962] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813966] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813969] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813972] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813975] [10676] 1012 10676 70480 5910 0 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813978] [10678] 1012 10678 69178 6232 1 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813981] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813984] [10700] 1012 10700 70080 5538 0 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813987] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813990] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813993] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.813997] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814000] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814003] [12116] 108 12116 11107 159 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814006] [12117] 108 12117 6816 94 1 0 0 scache
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814009] [12290] 108 12290 11107 160 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814012] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814015] [12556] 108 12556 11108 159 0 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814018] [12561] 108 12561 11108 160 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814021] [12566] 108 12566 11108 160 0 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814024] [12579] 108 12579 11108 160 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814027] [12603] 108 12603 11108 158 0 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814030] [12607] 108 12607 11108 161 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814033] [12743] 108 12743 11108 159 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814036] [12778] 108 12778 11108 158 0 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814039] [12782] 108 12782 11108 160 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814042] [12810] 108 12810 11108 159 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814045] [12816] 108 12816 11108 160 0 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814048] [12821] 108 12821 11108 161 1 0 0 smtp
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814051] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814054] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814057] [15505] 1012 15505 826248 763111 0 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814060] [15678] 1002 15678 18359 205 0 0 0 sshd
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814063] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814066] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814069] [16208] 1012 16208 155564 92268 1 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814072] [16210] 1012 16210 72944 9986 1 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814075] [16227] 108 16227 6792 69 0 0 0 pickup
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814078] [16282] 1012 16282 71382 8435 0 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814081] [16316] 1012 16316 1099 24 0 0 0 sh
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814084] [16317] 1012 16317 68825 5873 0 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814087] [16319] 1002 16319 3115 44 0 0 0 less
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814090] [16334] 1012 16334 1099 26 0 0 0 sh
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814093] [16335] 1012 16335 4651 61 0 0 0 git
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814096] [16336] 1012 16336 68476 5522 1 0 0 php
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814099] [16341] 1012 16341 1099 25 0 0 0 sh
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814102] [16342] 1012 16342 10901 175 0 0 0 ssh
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814105] Out of memory: Kill process 15505 (php) score 796 or sacrifice child
Jul 17 00:07:43 ip-10-61-174-46 kernel: [1546558.814116] Killed process 15505 (php) total-vm:3304992kB, anon-rss:3052444kB, file-rss:0kB
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116006] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116010] ntpd cpuset=/ mems_allowed=0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116013] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116015] Call Trace:
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116022] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116024] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116026] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116029] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116032] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116036] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116038] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116041] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116044] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116047] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116051] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116066] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116070] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116075] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116079] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116081] Mem-Info:
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116083] Node 0 DMA per-cpu:
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116085] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116087] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116089] Node 0 DMA32 per-cpu:
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116091] CPU 0: hi: 186, btch: 31 usd: 16
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116093] CPU 1: hi: 186, btch: 31 usd: 159
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116097] active_anon:930392 inactive_anon:1396 isolated_anon:0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116098] active_file:7 inactive_file:0 isolated_file:22
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116099] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116100] free:4025 slab_reclaimable:2787 slab_unreclaimable:3327
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116101] mapped:1352 shmem:1407 pagetables:4290 bounce:0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116103] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116111] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116116] Node 0 DMA32 free:7764kB min:7848kB low:9808kB high:11772kB active_anon:3721568kB inactive_anon:5584kB active_file:28kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5408kB shmem:5628kB slab_reclaimable:11148kB slab_unreclaimable:13308kB kernel_stack:1040kB pagetables:17160kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:585 all_unreclaimable? yes
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116124] lowmem_reserve[]: 0 0 0 0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116127] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116138] Node 0 DMA32: 1097*4kB 166*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7764kB
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116148] 1410 total pagecache pages
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116149] 0 pages in swap cache
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116151] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116152] Free swap = 0kB
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.116154] Total swap = 0kB
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121684] 985072 pages RAM
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121686] 24998 pages reserved
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121687] 9863 pages shared
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121688] 947084 pages non-shared
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121689] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121697] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121700] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121702] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121704] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121706] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121709] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121711] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121713] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121717] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121719] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121721] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121723] [ 975] 0 975 6276 79 1 0 0 master
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121726] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121728] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121731] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121733] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121735] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121737] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121739] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121742] [ 6563] 0 6563 89584 2519 0 0 0 php5-fpm
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121744] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121746] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121749] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121751] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121754] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121756] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121758] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121761] [10676] 1012 10676 70480 5918 1 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121763] [10678] 1012 10678 69178 6233 0 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121765] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121767] [10700] 1012 10700 70080 5538 0 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121769] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121772] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121774] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121776] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121778] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121781] [12116] 108 12116 11107 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121783] [12117] 108 12117 6816 93 1 0 0 scache
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121785] [12290] 108 12290 11107 160 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121787] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121789] [12556] 108 12556 11108 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121791] [12561] 108 12561 11108 160 0 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121793] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121795] [12579] 108 12579 11108 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121797] [12603] 108 12603 11108 158 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121799] [12607] 108 12607 11108 161 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121801] [12743] 108 12743 11108 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121803] [12778] 108 12778 11108 158 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121805] [12782] 108 12782 11108 159 0 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121808] [12810] 108 12810 11108 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121810] [12816] 108 12816 11108 159 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121812] [12821] 108 12821 11108 161 1 0 0 smtp
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121814] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121816] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121818] [15678] 1002 15678 18359 217 0 0 0 sshd
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121820] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121822] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121825] [16208] 1012 16208 649898 586732 0 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121827] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121829] [16319] 1002 16319 3115 63 1 0 0 less
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121831] [16346] 1012 16346 332157 268964 0 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121833] [16599] 1012 16599 76493 13506 1 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121835] [16687] 1012 16687 1099 24 0 0 0 sh
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121838] [16688] 1012 16688 68467 5515 1 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121840] [16689] 1012 16689 1099 24 0 0 0 sh
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121842] [16690] 1012 16690 68340 5425 1 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121844] [16691] 1012 16691 1099 25 0 0 0 sh
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121846] [16692] 1012 16692 68011 5050 0 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121848] [16696] 1012 16696 70480 5918 0 0 0 php
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121850] Out of memory: Kill process 16208 (php) score 612 or sacrifice child
Jul 17 00:08:06 ip-10-61-174-46 kernel: [1546582.121859] Killed process 16208 (php) total-vm:2599592kB, anon-rss:2346928kB, file-rss:0kB
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116023] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116027] ntpd cpuset=/ mems_allowed=0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116030] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116032] Call Trace:
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116039] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116041] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116043] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116046] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116051] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116073] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116077] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116080] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116084] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116088] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116091] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116094] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116098] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116102] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116104] Mem-Info:
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116106] Node 0 DMA per-cpu:
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116109] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116110] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116112] Node 0 DMA32 per-cpu:
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116114] CPU 0: hi: 186, btch: 31 usd: 43
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116116] CPU 1: hi: 186, btch: 31 usd: 23
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116120] active_anon:929875 inactive_anon:1396 isolated_anon:0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116121] active_file:6 inactive_file:4 isolated_file:14
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116122] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116123] free:4037 slab_reclaimable:2776 slab_unreclaimable:3419
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116124] mapped:1353 shmem:1407 pagetables:4669 bounce:0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116126] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116134] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116138] Node 0 DMA32 free:7812kB min:7848kB low:9808kB high:11772kB active_anon:3719500kB inactive_anon:5584kB active_file:24kB inactive_file:16kB unevictable:0kB isolated(anon):0kB isolated(file):56kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5412kB shmem:5628kB slab_reclaimable:11104kB slab_unreclaimable:13676kB kernel_stack:1096kB pagetables:18676kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:131 all_unreclaimable? yes
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116147] lowmem_reserve[]: 0 0 0 0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116150] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116160] Node 0 DMA32: 1446*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7832kB
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116170] 1442 total pagecache pages
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116172] 0 pages in swap cache
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116173] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116175] Free swap = 0kB
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.116176] Total swap = 0kB
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121705] 985072 pages RAM
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121708] 24998 pages reserved
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121709] 4073 pages shared
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121710] 952959 pages non-shared
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121712] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121721] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121725] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121727] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121730] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121732] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121734] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121737] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121739] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121742] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121744] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121747] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121749] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121751] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121753] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121756] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121758] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121760] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121762] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121764] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121767] [ 6563] 0 6563 89584 2508 1 0 0 php5-fpm
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121769] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121771] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121774] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121776] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121779] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121781] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121783] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121785] [10676] 1012 10676 70480 5927 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121787] [10678] 1012 10678 69178 6233 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121789] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121791] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121793] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121796] [12075] 108 12075 6795 75 1 0 0 trivial-rewrite
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121798] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121800] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121803] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121805] [12116] 108 12116 11107 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121807] [12117] 108 12117 6816 93 0 0 0 scache
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121809] [12290] 108 12290 11107 160 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121811] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121813] [12556] 108 12556 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121815] [12561] 108 12561 11108 160 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121817] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121819] [12579] 108 12579 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121821] [12603] 108 12603 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121823] [12607] 108 12607 11108 161 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121826] [12743] 108 12743 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121828] [12778] 108 12778 11108 158 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121830] [12782] 108 12782 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121832] [12810] 108 12810 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121834] [12816] 108 12816 11108 159 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121836] [12821] 108 12821 11108 161 1 0 0 smtp
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121838] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121840] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121842] [15678] 1002 15678 18397 221 1 0 0 sshd
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121844] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121847] [15948] 108 15948 6818 81 1 0 0 cleanup
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121849] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121851] [16319] 1002 16319 3115 63 1 0 0 less
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121853] [16346] 1012 16346 861470 798278 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121855] [16861] 1012 16861 79595 16617 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121857] [16997] 108 16997 6818 74 0 0 0 cleanup
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121859] [17023] 1012 17023 77068 14174 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121861] [17025] 1012 17025 71036 7999 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121863] [17026] 1012 17026 71164 8131 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121865] [17036] 1012 17036 1099 25 0 0 0 sh
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121867] [17037] 1012 17037 279828 861 0 0 0 git
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121869] [17038] 1012 17038 1099 26 0 0 0 sh
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121871] [17039] 1012 17039 279923 947 0 0 0 git
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121874] [17042] 1012 17042 74548 11612 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121876] [17046] 1012 17046 74099 11172 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121878] [17056] 1012 17056 1099 25 1 0 0 sh
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121880] [17057] 1012 17057 67622 4674 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121882] [17058] 1012 17058 1099 24 0 0 0 sh
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121884] [17059] 1012 17059 67622 4676 1 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121886] [17064] 1012 17064 68518 5521 0 0 0 php
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121888] Out of memory: Kill process 16346 (php) score 833 or sacrifice child
Jul 17 00:08:29 ip-10-61-174-46 kernel: [1546605.121897] Killed process 16346 (php) total-vm:3445880kB, anon-rss:3193112kB, file-rss:0kB
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898448] git invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898453] git cpuset=/ mems_allowed=0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898456] Pid: 17664, comm: git Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898458] Call Trace:
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898466] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898469] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898472] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898476] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898480] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898484] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898487] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898491] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898495] [<ffffffff8118691d>] ? do_lookup+0x14d/0x310
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898498] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898501] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898504] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898506] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898510] [<ffffffff8116369f>] ? kmem_cache_free+0x2f/0x110
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898513] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898516] [<ffffffff810e090d>] ? __call_rcu+0x9d/0x170
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898520] [<ffffffff81198ca0>] ? mntput_no_expire+0x30/0xf0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898521] [<ffffffff810e0a15>] ? call_rcu_sched+0x15/0x20
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898525] [<ffffffff81091dff>] ? __put_cred+0x3f/0x50
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898528] [<ffffffff811783d4>] ? sys_faccessat+0x114/0x1e0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898530] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898532] Mem-Info:
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898533] Node 0 DMA per-cpu:
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898535] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898537] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898538] Node 0 DMA32 per-cpu:
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898539] CPU 0: hi: 186, btch: 31 usd: 0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898541] CPU 1: hi: 186, btch: 31 usd: 1
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898543] active_anon:930353 inactive_anon:1396 isolated_anon:0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898544] active_file:0 inactive_file:26 isolated_file:0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898545] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898545] free:4048 slab_reclaimable:2914 slab_unreclaimable:3352
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898546] mapped:1354 shmem:1407 pagetables:4183 bounce:0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898548] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898554] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898557] Node 0 DMA32 free:7856kB min:7848kB low:9808kB high:11772kB active_anon:3721412kB inactive_anon:5584kB active_file:0kB inactive_file:104kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5628kB slab_reclaimable:11656kB slab_unreclaimable:13408kB kernel_stack:1056kB pagetables:16732kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:177 all_unreclaimable? yes
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898564] lowmem_reserve[]: 0 0 0 0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898566] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898573] Node 0 DMA32: 1464*4kB 10*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7984kB
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898579] 1484 total pagecache pages
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898580] 0 pages in swap cache
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898582] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898583] Free swap = 0kB
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.898584] Total swap = 0kB
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904085] 985072 pages RAM
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904088] 24998 pages reserved
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904089] 4098 pages shared
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904090] 952967 pages non-shared
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904092] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904100] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904104] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904107] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904110] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904114] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904117] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904120] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904123] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904127] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904131] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904134] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904137] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904140] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904143] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904146] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904150] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904153] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904156] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904159] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904162] [ 6563] 0 6563 89584 2537 0 0 0 php5-fpm
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904166] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904169] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904172] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904175] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904179] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904182] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904185] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904188] [10676] 1012 10676 70480 5938 0 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904191] [10678] 1012 10678 69178 6238 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904194] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904197] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904200] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904203] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904206] [12077] 108 12077 11107 161 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904210] [12107] 108 12107 11107 159 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904213] [12108] 108 12108 11107 161 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904216] [12116] 108 12116 11107 160 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904219] [12117] 108 12117 6816 93 1 0 0 scache
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904222] [12290] 108 12290 11107 160 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904225] [12312] 108 12312 11108 158 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904228] [12556] 108 12556 11108 159 1 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904231] [12561] 108 12561 11108 160 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904234] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904237] [12579] 108 12579 11108 160 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904240] [12603] 108 12603 11108 158 1 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904243] [12607] 108 12607 11108 161 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904246] [12743] 108 12743 11108 160 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904249] [12778] 108 12778 11108 158 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904252] [12782] 108 12782 11108 159 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904255] [12810] 108 12810 11108 160 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904258] [12816] 108 12816 11108 159 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904261] [12821] 108 12821 11108 161 0 0 0 smtp
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904264] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904267] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904270] [15678] 1002 15678 18397 253 1 0 0 sshd
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904273] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904276] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904279] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904282] [16319] 1002 16319 3115 63 0 0 0 less
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904285] [16997] 108 16997 6818 82 0 0 0 cleanup
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904288] [17023] 1012 17023 646554 583333 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904291] [17415] 1012 17415 340491 277431 0 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904294] [17548] 1012 17548 75544 12603 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904297] [17656] 1012 17656 1099 25 1 0 0 sh
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904300] [17657] 1012 17657 68825 5877 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904302] [17663] 1012 17663 1099 25 0 0 0 sh
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904306] [17664] 1012 17664 5218 84 0 0 0 git
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904308] [17665] 1012 17665 68476 5522 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904311] [17667] 1012 17667 1099 24 0 0 0 sh
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904314] [17668] 1012 17668 10901 166 1 0 0 ssh
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904317] [17669] 1012 17669 11724 64 1 0 0 php
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904320] Out of memory: Kill process 17023 (php) score 608 or sacrifice child
Jul 17 00:08:55 ip-10-61-174-46 kernel: [1546630.904330] Killed process 17023 (php) total-vm:2586216kB, anon-rss:2333332kB, file-rss:0kB
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515068] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515073] php cpuset=/ mems_allowed=0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515076] Pid: 17915, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515078] Call Trace:
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515087] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515090] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515094] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515098] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515102] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515107] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515110] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515114] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515119] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515122] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515127] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515130] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515133] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515136] [<ffffffff81004132>] ? load_TLS_descriptor+0x42/0xd0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515139] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515143] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515148] [<ffffffff81013878>] ? __switch_to+0x288/0x360
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515152] [<ffffffff81056803>] ? finish_task_switch+0x53/0xf0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515156] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515159] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515162] Mem-Info:
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515163] Node 0 DMA per-cpu:
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515166] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515167] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515169] Node 0 DMA32 per-cpu:
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515171] CPU 0: hi: 186, btch: 31 usd: 191
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515173] CPU 1: hi: 186, btch: 31 usd: 160
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515176] active_anon:927545 inactive_anon:1396 isolated_anon:0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515177] active_file:163 inactive_file:145 isolated_file:32
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515178] unevictable:0 dirty:25 writeback:0 unstable:0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515179] free:3862 slab_reclaimable:2655 slab_unreclaimable:3873
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515180] mapped:1495 shmem:1407 pagetables:4727 bounce:0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515182] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515191] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515195] Node 0 DMA32 free:7112kB min:7848kB low:9808kB high:11772kB active_anon:3710180kB inactive_anon:5584kB active_file:652kB inactive_file:580kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:3862656kB mlocked:0kB dirty:100kB writeback:0kB mapped:5980kB shmem:5628kB slab_reclaimable:10620kB slab_unreclaimable:15492kB kernel_stack:1112kB pagetables:18908kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:32 all_unreclaimable? no
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515204] lowmem_reserve[]: 0 0 0 0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515207] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515217] Node 0 DMA32: 1124*4kB 21*8kB 45*16kB 2*32kB 0*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 7112kB
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515226] 1802 total pagecache pages
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515228] 0 pages in swap cache
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515229] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515231] Free swap = 0kB
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.515232] Total swap = 0kB
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521489] 985072 pages RAM
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521492] 24998 pages reserved
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521493] 5493 pages shared
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521494] 952656 pages non-shared
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521496] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521504] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521508] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521511] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521514] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521517] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521521] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521524] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521527] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521531] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521534] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521537] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521541] [ 975] 0 975 6276 79 1 0 0 master
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521544] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521547] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521550] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521553] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521556] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521559] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521562] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521566] [ 6563] 0 6563 89584 2511 0 0 0 php5-fpm
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521569] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521572] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521575] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521579] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521582] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521585] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521588] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521591] [10676] 1012 10676 70480 6012 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521594] [10678] 1012 10678 69178 6307 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521597] [10680] 1012 10680 71906 9055 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521600] [10700] 1012 10700 70080 5550 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521603] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521606] [12075] 108 12075 6795 75 1 0 0 trivial-rewrite
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521609] [12077] 108 12077 11107 160 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521612] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521616] [12108] 108 12108 11107 160 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521619] [12116] 108 12116 11107 159 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521622] [12117] 108 12117 6816 93 1 0 0 scache
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521625] [12290] 108 12290 11107 160 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521628] [12312] 108 12312 11108 158 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521631] [12556] 108 12556 11108 159 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521634] [12561] 108 12561 11108 160 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521637] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521640] [12579] 108 12579 11108 159 1 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521643] [12603] 108 12603 11108 158 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521646] [12607] 108 12607 11108 161 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521649] [12743] 108 12743 11108 160 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521652] [12778] 108 12778 11108 158 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521655] [12782] 108 12782 11108 159 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521658] [12810] 108 12810 11108 160 1 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521661] [12816] 108 12816 11108 159 1 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521664] [12821] 108 12821 11108 161 0 0 0 smtp
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521667] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521670] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521673] [15678] 1002 15678 18397 253 0 0 0 sshd
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521676] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521679] [15948] 108 15948 6818 81 1 0 0 cleanup
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521682] [16227] 108 16227 6792 75 1 0 0 pickup
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521685] [16319] 1002 16319 3115 63 0 0 0 less
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521688] [16997] 108 16997 6818 82 1 0 0 cleanup
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521691] [17415] 1012 17415 813865 750788 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521694] [17856] 1012 17856 71625 8807 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521697] [17864] 1012 17864 141450 75856 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521700] [17869] 1012 17869 71372 8494 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521703] [17883] 1012 17883 1099 26 0 0 0 sh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521706] [17884] 1012 17884 68825 5960 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521709] [17887] 1012 17887 1099 26 1 0 0 sh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521712] [17888] 1012 17888 68825 5961 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521715] [17894] 1012 17894 71089 8182 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521718] [17900] 1012 17900 1099 25 0 0 0 sh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521721] [17901] 1012 17901 4724 58 0 0 0 git
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521724] [17902] 1012 17902 68476 5607 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521727] [17904] 1012 17904 1099 27 0 0 0 sh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521730] [17905] 1012 17905 68610 5708 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521733] [17911] 1012 17911 1099 26 0 0 0 sh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521736] [17912] 1012 17912 5065 69 1 0 0 git
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521739] [17913] 1012 17913 66093 3726 1 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521742] [17915] 1012 17915 66188 3799 0 0 0 php
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521745] [17917] 1012 17917 1099 24 1 0 0 sh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521747] [17918] 1012 17918 8210 39 0 0 0 ssh
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521750] Out of memory: Kill process 17415 (php) score 783 or sacrifice child
Jul 17 00:09:12 ip-10-61-174-46 kernel: [1546648.521762] Killed process 17415 (php) total-vm:3255460kB, anon-rss:3002960kB, file-rss:192kB
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796150] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796154] php cpuset=/ mems_allowed=0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796157] Pid: 17864, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796159] Call Trace:
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796166] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796168] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796170] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796173] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796176] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796180] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796182] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796185] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796189] [<ffffffff81668c1b>] ? xen_hypervisor_callback+0x1b/0x20
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796192] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796195] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796198] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796200] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796203] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796205] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796207] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796209] [<ffffffff81004dd2>] ? xen_mc_flush+0xb2/0x1c0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796211] [<ffffffff8100478d>] ? xen_clts+0x8d/0x190
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796215] [<ffffffff8165c00c>] ? __schedule+0x3cc/0x6f0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796218] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796219] Mem-Info:
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796221] Node 0 DMA per-cpu:
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796223] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796224] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796225] Node 0 DMA32 per-cpu:
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796227] CPU 0: hi: 186, btch: 31 usd: 0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796228] CPU 1: hi: 186, btch: 31 usd: 0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796230] active_anon:930037 inactive_anon:1396 isolated_anon:0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796231] active_file:16 inactive_file:7 isolated_file:0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796232] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796232] free:2834 slab_reclaimable:2644 slab_unreclaimable:3696
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796233] mapped:1341 shmem:1407 pagetables:4384 bounce:0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796235] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796241] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796244] Node 0 DMA32 free:3000kB min:7848kB low:9808kB high:11772kB active_anon:3720148kB inactive_anon:5584kB active_file:64kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5364kB shmem:5628kB slab_reclaimable:10576kB slab_unreclaimable:14784kB kernel_stack:1048kB pagetables:17536kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10092 all_unreclaimable? yes
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796251] lowmem_reserve[]: 0 0 0 0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796253] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796260] Node 0 DMA32: 270*4kB 17*8kB 1*16kB 0*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 3088kB
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796267] 1461 total pagecache pages
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796268] 0 pages in swap cache
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796269] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796270] Free swap = 0kB
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.796271] Total swap = 0kB
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801783] 985072 pages RAM
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801785] 24998 pages reserved
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801786] 4091 pages shared
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801787] 954199 pages non-shared
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801789] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801799] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801802] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801805] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801808] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801812] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801815] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801818] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801822] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801826] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801829] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801832] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801836] [ 975] 0 975 6276 79 1 0 0 master
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801839] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801842] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801845] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801848] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801851] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801854] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801857] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801861] [ 6563] 0 6563 89584 2495 1 0 0 php5-fpm
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801864] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801867] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801870] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801874] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801877] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801880] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801883] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801886] [10676] 1012 10676 70480 5956 0 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801889] [10678] 1012 10678 69178 6239 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801892] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801895] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801898] [10701] 1012 10701 68618 5626 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801902] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801905] [12077] 108 12077 11107 160 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801908] [12107] 108 12107 11107 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801911] [12108] 108 12108 11107 160 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801914] [12116] 108 12116 11107 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801917] [12117] 108 12117 6816 93 0 0 0 scache
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801920] [12290] 108 12290 11107 160 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801923] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801926] [12556] 108 12556 11108 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801929] [12561] 108 12561 11108 160 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801932] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801935] [12579] 108 12579 11108 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801938] [12603] 108 12603 11108 158 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801941] [12607] 108 12607 11108 161 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801944] [12743] 108 12743 11108 160 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801947] [12778] 108 12778 11108 158 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801950] [12782] 108 12782 11108 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801953] [12810] 108 12810 11108 160 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801956] [12816] 108 12816 11108 159 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801959] [12821] 108 12821 11108 161 0 0 0 smtp
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801962] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801965] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801968] [15678] 1002 15678 18397 253 1 0 0 sshd
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801971] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801974] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801977] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801980] [16319] 1002 16319 3115 63 1 0 0 less
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801983] [16997] 108 16997 6818 82 0 0 0 cleanup
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801986] [17864] 1012 17864 831950 768832 0 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801989] [18146] 1012 18146 142354 75975 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801992] [18148] 1012 18148 71820 8906 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801995] [18152] 1012 18152 71372 8392 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.801998] [18181] 1012 18181 69901 6912 0 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802001] [18186] 1012 18186 68978 6028 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802004] [18188] 1012 18188 1099 25 0 0 0 sh
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802007] [18189] 1012 18189 67817 4887 0 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802010] [18190] 1012 18190 1099 25 0 0 0 sh
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802013] [18191] 1012 18191 67817 4887 1 0 0 php
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802015] Out of memory: Kill process 17864 (php) score 802 or sacrifice child
Jul 17 00:09:34 ip-10-61-174-46 kernel: [1546669.802027] Killed process 17864 (php) total-vm:3327800kB, anon-rss:3075328kB, file-rss:0kB
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816717] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816721] php cpuset=/ mems_allowed=0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816724] Pid: 18455, comm: php Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816726] Call Trace:
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816732] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816735] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816737] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816740] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816743] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816746] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816749] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816752] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816755] [<ffffffff8112545d>] ? lru_cache_add_lru+0x2d/0x50
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816757] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816760] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816763] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816765] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816768] [<ffffffff8100a63d>] ? xen_force_evtchn_callback+0xd/0x10
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816770] [<ffffffff8100ada2>] ? check_events+0x12/0x20
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816773] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816776] [<ffffffff81144d28>] ? do_mmap_pgoff+0x348/0x360
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816779] [<ffffffff81144e21>] ? sys_mmap_pgoff+0xe1/0x230
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816782] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816783] Mem-Info:
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816784] Node 0 DMA per-cpu:
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816786] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816788] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816789] Node 0 DMA32 per-cpu:
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816790] CPU 0: hi: 186, btch: 31 usd: 0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816792] CPU 1: hi: 186, btch: 31 usd: 0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816794] active_anon:929664 inactive_anon:1396 isolated_anon:0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816795] active_file:0 inactive_file:22 isolated_file:0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816796] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816796] free:3194 slab_reclaimable:2490 slab_unreclaimable:3792
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816797] mapped:1356 shmem:1407 pagetables:4272 bounce:0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816798] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816805] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816808] Node 0 DMA32 free:4440kB min:7848kB low:9808kB high:11772kB active_anon:3718656kB inactive_anon:5584kB active_file:0kB inactive_file:88kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5424kB shmem:5628kB slab_reclaimable:9960kB slab_unreclaimable:15168kB kernel_stack:1064kB pagetables:17088kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:135 all_unreclaimable? yes
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816814] lowmem_reserve[]: 0 0 0 0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816817] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816823] Node 0 DMA32: 602*4kB 15*8kB 0*16kB 0*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 4448kB
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816830] 1429 total pagecache pages
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816831] 0 pages in swap cache
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816832] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816834] Free swap = 0kB
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.816834] Total swap = 0kB
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822332] 985072 pages RAM
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822334] 24998 pages reserved
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822336] 4076 pages shared
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822337] 953872 pages non-shared
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822339] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822347] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822351] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822354] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822357] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822360] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822364] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822367] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822371] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822374] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822378] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822381] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822384] [ 975] 0 975 6276 79 1 0 0 master
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822387] [ 983] 108 983 6850 124 1 0 0 qmgr
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822391] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822394] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822397] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822400] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822403] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822406] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822410] [ 6563] 0 6563 89584 2536 1 0 0 php5-fpm
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822413] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822416] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822419] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822423] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822426] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822429] [10540] 1002 10540 18359 204 1 0 0 sshd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822432] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822435] [10676] 1012 10676 70544 5968 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822438] [10678] 1012 10678 69178 6239 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822441] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822444] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822447] [10701] 1012 10701 68618 5675 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822450] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822453] [12077] 108 12077 11107 160 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822457] [12107] 108 12107 11107 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822460] [12108] 108 12108 11107 160 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822463] [12116] 108 12116 11107 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822466] [12117] 108 12117 6816 93 0 0 0 scache
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822469] [12290] 108 12290 11107 160 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822472] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822475] [12556] 108 12556 11108 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822478] [12561] 108 12561 11108 160 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822480] [12566] 108 12566 11108 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822484] [12579] 108 12579 11108 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822487] [12603] 108 12603 11108 158 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822490] [12607] 108 12607 11108 161 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822493] [12743] 108 12743 11108 160 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822496] [12778] 108 12778 11108 158 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822499] [12782] 108 12782 11108 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822502] [12810] 108 12810 11108 160 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822504] [12816] 108 12816 11108 159 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822507] [12821] 108 12821 11108 161 0 0 0 smtp
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822510] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822513] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822516] [15678] 1002 15678 18397 253 1 0 0 sshd
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822519] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822522] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822525] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822528] [16319] 1002 16319 3115 63 1 0 0 less
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822531] [16997] 108 16997 6818 82 0 0 0 cleanup
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822534] [18146] 1012 18146 830562 767364 0 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822537] [18359] 1012 18359 153073 89777 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822540] [18403] 1012 18403 71691 8744 1 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822543] [18442] 1012 18442 1099 24 0 0 0 sh
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822546] [18443] 1012 18443 68825 5878 0 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822549] [18451] 1012 18451 1099 25 0 0 0 sh
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822552] [18452] 1012 18452 5218 68 1 0 0 git
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822555] [18453] 1012 18453 68476 5523 0 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822558] [18455] 1012 18455 69648 6632 0 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822561] [18457] 1012 18457 1099 25 0 0 0 sh
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822564] [18458] 1012 18458 10901 165 1 0 0 ssh
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822567] [18459] 1012 18459 11718 117 0 0 0 php
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822570] Out of memory: Kill process 18146 (php) score 800 or sacrifice child
Jul 17 00:09:56 ip-10-61-174-46 kernel: [1546691.822581] Killed process 18146 (php) total-vm:3322248kB, anon-rss:3069456kB, file-rss:0kB
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.115999] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116003] ntpd cpuset=/ mems_allowed=0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116005] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116007] Call Trace:
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116014] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116017] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116019] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116022] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116025] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116029] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116031] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116034] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116036] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116040] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116042] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116045] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116049] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116068] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116073] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116075] Mem-Info:
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116076] Node 0 DMA per-cpu:
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116079] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116081] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116082] Node 0 DMA32 per-cpu:
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116085] CPU 0: hi: 186, btch: 31 usd: 0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116087] CPU 1: hi: 186, btch: 31 usd: 55
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116090] active_anon:928430 inactive_anon:1396 isolated_anon:0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116091] active_file:6 inactive_file:13 isolated_file:22
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116092] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116093] free:3885 slab_reclaimable:2691 slab_unreclaimable:3841
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116094] mapped:1360 shmem:1407 pagetables:4505 bounce:0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116096] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116105] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116109] Node 0 DMA32 free:7204kB min:7848kB low:9808kB high:11772kB active_anon:3713720kB inactive_anon:5584kB active_file:24kB inactive_file:52kB unevictable:0kB isolated(anon):0kB isolated(file):88kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5440kB shmem:5628kB slab_reclaimable:10764kB slab_unreclaimable:15364kB kernel_stack:1080kB pagetables:18020kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:78 all_unreclaimable? yes
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116117] lowmem_reserve[]: 0 0 0 0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116121] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116131] Node 0 DMA32: 1151*4kB 86*8kB 1*16kB 1*32kB 1*64kB 0*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 7196kB
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116141] 1443 total pagecache pages
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116143] 0 pages in swap cache
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116144] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116146] Free swap = 0kB
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.116147] Total swap = 0kB
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121716] 985072 pages RAM
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121718] 24998 pages reserved
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121719] 4074 pages shared
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121720] 953121 pages non-shared
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121722] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121729] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121732] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121734] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121736] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121738] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121741] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121743] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121746] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121749] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121751] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121753] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121756] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121758] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121760] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121762] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121764] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121767] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121769] [ 1242] 107 1242 9445 162 0 0 0 ntpd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121771] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121773] [ 6563] 0 6563 89584 2520 1 0 0 php5-fpm
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121775] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121778] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121780] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121783] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121785] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121787] [10540] 1002 10540 18359 214 0 0 0 sshd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121789] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121792] [10676] 1012 10676 70544 5980 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121794] [10678] 1012 10678 69242 6243 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121796] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121798] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121800] [10701] 1012 10701 68618 5675 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121802] [12075] 108 12075 6795 75 0 0 0 trivial-rewrite
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121804] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121806] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121809] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121811] [12116] 108 12116 11107 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121813] [12117] 108 12117 6816 93 1 0 0 scache
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121815] [12290] 108 12290 11107 160 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121817] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121819] [12556] 108 12556 11108 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121821] [12561] 108 12561 11108 160 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121823] [12566] 108 12566 11108 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121826] [12579] 108 12579 11108 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121828] [12603] 108 12603 11108 158 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121830] [12607] 108 12607 11108 161 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121832] [12743] 108 12743 11108 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121834] [12778] 108 12778 11108 158 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121836] [12782] 108 12782 11108 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121838] [12810] 108 12810 11108 160 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121840] [12816] 108 12816 11108 159 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121842] [12821] 108 12821 11108 161 1 0 0 smtp
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121844] [14454] 1002 14454 3115 63 0 0 0 less
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121846] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121848] [15678] 1002 15678 18397 253 1 0 0 sshd
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121850] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121853] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121855] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121857] [16319] 1002 16319 3115 63 1 0 0 less
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121859] [16997] 108 16997 6818 82 0 0 0 cleanup
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121861] [18359] 1012 18359 801248 738125 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121863] [18728] 1012 18728 154020 90533 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121865] [18858] 1012 18858 71872 8947 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121868] [18867] 1012 18867 71425 8469 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121870] [18868] 1012 18868 71690 8725 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121872] [18875] 1012 18875 72269 9334 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121874] [18877] 1012 18877 1099 24 1 0 0 sh
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121876] [18878] 1012 18878 68825 5873 1 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121878] [18880] 1012 18880 70298 7331 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121880] [18886] 1012 18886 1099 25 0 0 0 sh
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121882] [18887] 1012 18887 4738 66 0 0 0 git
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121884] [18888] 1012 18888 68476 5523 0 0 0 php
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121886] [18890] 1012 18890 1099 25 0 0 0 sh
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121888] [18891] 1012 18891 10901 175 0 0 0 ssh
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121890] Out of memory: Kill process 18359 (php) score 770 or sacrifice child
Jul 17 00:10:22 ip-10-61-174-46 kernel: [1546718.121900] Killed process 18359 (php) total-vm:3204992kB, anon-rss:2952500kB, file-rss:0kB
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.115988] ntpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.115992] ntpd cpuset=/ mems_allowed=0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.115995] Pid: 1242, comm: ntpd Not tainted 3.2.0-75-virtual #110-Ubuntu
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.115997] Call Trace:
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116004] [<ffffffff8111b9e1>] dump_header+0x91/0xe0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116006] [<ffffffff8111bd95>] oom_kill_process+0x85/0xb0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116008] [<ffffffff8111c13a>] out_of_memory+0xfa/0x220
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116011] [<ffffffff81121b13>] __alloc_pages_nodemask+0x8c3/0x8e0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116014] [<ffffffff81159056>] alloc_pages_current+0xb6/0x120
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116018] [<ffffffff81118907>] __page_cache_alloc+0xb7/0xd0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116020] [<ffffffff8111a8d2>] filemap_fault+0x212/0x3c0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116023] [<ffffffff8113b2a2>] __do_fault+0x72/0x550
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116025] [<ffffffff8113e95a>] handle_pte_fault+0xfa/0x200
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116029] [<ffffffff810063fe>] ? xen_pmd_val+0xe/0x10
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116031] [<ffffffff81005379>] ? __raw_callee_save_xen_pmd_val+0x11/0x1e
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116033] [<ffffffff8113fc39>] handle_mm_fault+0x269/0x370
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116037] [<ffffffff81661e74>] do_page_fault+0x184/0x550
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116040] [<ffffffff8103ecd5>] ? pvclock_clocksource_read+0x55/0xf0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116043] [<ffffffff8165ea95>] page_fault+0x25/0x30
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116045] Mem-Info:
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116046] Node 0 DMA per-cpu:
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116048] CPU 0: hi: 0, btch: 1 usd: 0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116051] CPU 1: hi: 0, btch: 1 usd: 0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116064] Node 0 DMA32 per-cpu:
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116067] CPU 0: hi: 186, btch: 31 usd: 82
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116068] CPU 1: hi: 186, btch: 31 usd: 72
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116072] active_anon:931267 inactive_anon:1396 isolated_anon:0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116073] active_file:0 inactive_file:6 isolated_file:14
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116073] unevictable:0 dirty:0 writeback:0 unstable:0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116074] free:4042 slab_reclaimable:2448 slab_unreclaimable:3330
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116075] mapped:1354 shmem:1407 pagetables:3835 bounce:0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116078] Node 0 DMA free:8336kB min:16kB low:20kB high:24kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:8080kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116086] lowmem_reserve[]: 0 3772 3772 3772
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116090] Node 0 DMA32 free:7832kB min:7848kB low:9808kB high:11772kB active_anon:3725068kB inactive_anon:5584kB active_file:0kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):56kB present:3862656kB mlocked:0kB dirty:0kB writeback:0kB mapped:5416kB shmem:5628kB slab_reclaimable:9792kB slab_unreclaimable:13320kB kernel_stack:968kB pagetables:15340kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:264 all_unreclaimable? yes
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116098] lowmem_reserve[]: 0 0 0 0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116102] Node 0 DMA: 2*4kB 3*8kB 1*16kB 1*32kB 3*64kB 3*128kB 2*256kB 0*512kB 1*1024kB 1*2048kB 1*4096kB = 8336kB
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116112] Node 0 DMA32: 926*4kB 260*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 7832kB
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116122] 1456 total pagecache pages
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116124] 0 pages in swap cache
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116125] Swap cache stats: add 0, delete 0, find 0/0
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116127] Free swap = 0kB
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.116128] Total swap = 0kB
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121587] 985072 pages RAM
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121589] 24998 pages reserved
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121590] 4068 pages shared
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121591] 952892 pages non-shared
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121593] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121601] [ 475] 0 475 4307 51 0 0 0 upstart-udev-br
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121603] [ 477] 0 477 5393 119 1 -17 -1000 udevd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121605] [ 530] 0 530 5363 105 1 -17 -1000 udevd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121608] [ 531] 0 531 5363 103 1 -17 -1000 udevd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121610] [ 607] 0 607 3796 46 0 0 0 upstart-socket-
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121613] [ 664] 0 664 1815 124 0 0 0 dhclient3
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121615] [ 765] 0 765 12508 152 1 -17 -1000 sshd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121617] [ 778] 102 778 5954 71 0 0 0 dbus-daemon
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121621] [ 790] 101 790 63525 802 0 0 0 rsyslogd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121623] [ 827] 0 827 1081 36 1 0 0 acpid
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121625] [ 831] 0 831 4226 41 0 0 0 atd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121628] [ 975] 0 975 6276 79 0 0 0 master
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121630] [ 983] 108 983 6850 124 0 0 0 qmgr
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121632] [ 1024] 0 1024 3309 38 0 0 0 mdadm
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121634] [ 1049] 0 1049 1072 23 1 0 0 collectdmon
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121637] [ 1051] 0 1051 201625 844 0 0 0 collectd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121639] [ 1088] 0 1088 3625 40 1 0 0 getty
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121641] [ 1242] 107 1242 9445 162 1 0 0 ntpd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121643] [15657] 0 15657 4777 59 0 0 0 cron
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121645] [ 6563] 0 6563 89584 2491 0 0 0 php5-fpm
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121648] [ 6601] 33 6601 89584 1189 1 0 0 php5-fpm
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121650] [ 6602] 33 6602 89584 1189 1 0 0 php5-fpm
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121653] [28388] 1002 28388 6462 147 0 0 0 screen
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121655] [28389] 1002 28389 5559 765 1 0 0 bash
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121657] [10382] 0 10382 18359 200 0 0 0 sshd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121659] [10540] 1002 10540 18359 214 0 0 0 sshd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121662] [10541] 1002 10541 6544 1751 0 0 0 bash
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121664] [10676] 1012 10676 70544 5995 0 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121666] [10678] 1012 10678 69242 6243 1 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121668] [10680] 1012 10680 71906 8989 1 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121670] [10700] 1012 10700 70080 5538 1 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121672] [10701] 1012 10701 68618 5683 1 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121674] [12075] 108 12075 6795 75 1 0 0 trivial-rewrite
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121676] [12077] 108 12077 11107 160 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121679] [12107] 108 12107 11107 159 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121681] [12108] 108 12108 11107 160 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121683] [12116] 108 12116 11107 159 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121685] [12117] 108 12117 6816 93 1 0 0 scache
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121688] [12290] 108 12290 11107 160 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121689] [12312] 108 12312 11108 158 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121691] [12556] 108 12556 11108 159 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121693] [12561] 108 12561 11108 160 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121695] [12566] 108 12566 11108 159 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121697] [12603] 108 12603 11108 158 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121700] [12607] 108 12607 11108 161 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121702] [12778] 108 12778 11108 158 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121704] [12782] 108 12782 11108 159 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121706] [12816] 108 12816 11108 159 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121708] [12821] 108 12821 11108 161 1 0 0 smtp
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121710] [14454] 1002 14454 3115 63 1 0 0 less
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121712] [15470] 0 15470 18359 200 0 0 0 sshd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121714] [15678] 1002 15678 18397 253 1 0 0 sshd
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121717] [15685] 1002 15685 6543 1749 1 0 0 bash
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121719] [15948] 108 15948 6818 81 0 0 0 cleanup
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121721] [16227] 108 16227 6792 75 0 0 0 pickup
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121723] [16319] 1002 16319 3115 63 1 0 0 less
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121725] [16997] 108 16997 6818 82 0 0 0 cleanup
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121727] [18728] 1012 18728 937915 874809 0 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121729] [19109] 1012 19109 71949 8932 0 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121731] [19178] 1012 19178 65218 2757 0 0 0 php
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121733] Out of memory: Kill process 18728 (php) score 913 or sacrifice child
Jul 17 00:10:48 ip-10-61-174-46 kernel: [1546744.121743] Killed process 18728 (php) total-vm:3751660kB, anon-rss:3499236kB, file-rss:0kB

File Metadata

Mime Type
text/plain
Storage Engine
amazon-s3
Storage Format
Raw Data
Storage Handle
phabricator/ic/rb/fjoxifzhbgae2gef
Default Alt Text
kern.log (4 MB)

Event Timeline

Phabricator · Built by Phacility