java - centos服务器CPU被GC task占满,程序无内存溢出
PHP中文网
PHP中文网 2017-04-18 10:31:17
0
0
1087
  1. 环境:服务器为centos,2个8核16线CPU,内存64G,jdk版本1.7.0_67

  2. 问题描述:服务器部署的java应用,jvm堆内存为8G,启动一段时间后CPU占用飙升到2300%

    • jstack查看获得如下信息:

      "VM Thread" prio=10 tid=0x00007f5ac4095800 nid=0xa599 runnable
      
      "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007f5ac4021000 nid=0xa56b runnable
      "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007f5ac4023000 nid=0xa56c runnable
      "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007f5ac4025000 nid=0xa56d runnable
      "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007f5ac4026800 nid=0xa56e runnable
      "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007f5ac4028800 nid=0xa56f runnable
      "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007f5ac402a800 nid=0xa570 runnable
      "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007f5ac402c800 nid=0xa571 runnable
      "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007f5ac402e800 nid=0xa572 runnable
      "GC task thread#8 (ParallelGC)" prio=10 tid=0x00007f5ac4030800 nid=0xa573 runnable
      "GC task thread#9 (ParallelGC)" prio=10 tid=0x00007f5ac4032800 nid=0xa574 runnable
      "GC task thread#10 (ParallelGC)" prio=10 tid=0x00007f5ac4034000 nid=0xa575 runnable
      "GC task thread#11 (ParallelGC)" prio=10 tid=0x00007f5ac4036000 nid=0xa576 runnable
      "GC task thread#12 (ParallelGC)" prio=10 tid=0x00007f5ac4038000 nid=0xa577 runnable
      "GC task thread#13 (ParallelGC)" prio=10 tid=0x00007f5ac403a000 nid=0xa578 runnable
      "GC task thread#14 (ParallelGC)" prio=10 tid=0x00007f5ac403c000 nid=0xa579 runnable
      "GC task thread#15 (ParallelGC)" prio=10 tid=0x00007f5ac403e000 nid=0xa57a runnable
      "GC task thread#16 (ParallelGC)" prio=10 tid=0x00007f5ac403f800 nid=0xa57b runnable
      "GC task thread#17 (ParallelGC)" prio=10 tid=0x00007f5ac4041800 nid=0xa57c runnable
      "GC task thread#18 (ParallelGC)" prio=10 tid=0x00007f5ac4043800 nid=0xa57d runnable
      "GC task thread#19 (ParallelGC)" prio=10 tid=0x00007f5ac4045800 nid=0xa57e runnable
      "GC task thread#20 (ParallelGC)" prio=10 tid=0x00007f5ac4047800 nid=0xa57f runnable
      "GC task thread#21 (ParallelGC)" prio=10 tid=0x00007f5ac4049800 nid=0xa580 runnable
      "GC task thread#22 (ParallelGC)" prio=10 tid=0x00007f5ac404b800 nid=0xa581 runnable
      "VM Periodic Task Thread" prio=10 tid=0x00007f5ac40d5800 nid=0xa5a3 waiting on condition
      
      JNI global references: 760
    • heap信息如下:

       PSYoungGen      
      total 5208064K, used 1186489K [0x00000006b1000000, 0x0000000800000000, 0x0000000800000000)
      eden space 4918272K, 20% used [0x00000006b1000000,0x00000006ef010b20,0x00000007dd300000)
      from space 289792K, 58% used [0x00000007ee500000,0x00000007f8b9db50,0x0000000800000000)
      to   space 280576K, 0% used [0x00000007dd300000,0x00000007dd300000,0x00000007ee500000)
       ParOldGen       
      total 1714688K, used 705292K [0x0000000413000000, 0x000000047ba80000, 0x00000006b1000000)
      object space 1714688K, 41% used [0x0000000413000000,0x000000043e0c3378,0x000000047ba80000)
    • dump下来6G多的内存快照用MAT做分析,疑似溢出只有38Mb,看了一下是设计的阻塞队列,基本可以排除内存溢出

    • 出问题部分的代码并行2个线程,一个处理json的序列化与反序列化以及zip流的写出操作,json用的是jackson2.3,zip流方面检查过了没有资源未关闭导致溢出的情况;还有一个对处理完的数据进行推送,收集到指定数量或等待指定时间才做一次推送

    现在能想到可能的情况都做了排查,本机单独运行出问题的模块也没什么问题,有没有遇到过类似问题的朋友,还望不吝赐教...

PHP中文网
PHP中文网

认证高级PHP讲师

membalas semua(0)
Muat turun terkini
Lagi>
kesan web
Kod sumber laman web
Bahan laman web
Templat hujung hadapan