环境:linux redhat 2.6.18-194.el5
JDK: 1.6.0_27 x64
btrace版本: 1.2.3
tomcat:7.0.22
场景:
把btrace-agent加载到tomcat中启动, 并且只加载一个Btrace脚本文件,启动时出现死锁,而且是每次必现。
Thread dump如下:
Found one Java-level deadlock:
=============================
"pool-1-thread-1":
waiting to lock monitor 0x000000005fcf5680 (object 0x00000007590e6b48, a sun.misc.Launcher$AppClassLoader),
which is held by "main"
"main":
waiting to lock monitor 0x000000005fcf4180 (object 0x000000060644f890, a <unknown>),
which is held by "pool-1-thread-1"
Java stack information for the threads listed above:
===================================================
"pool-1-thread-1":
at com.sun.btrace.runtime.ClassFilter.isCandidate(ClassFilter.java:158)
at com.sun.btrace.runtime.ClassFilter.isCandidate(ClassFilter.java:145)
at com.sun.btrace.agent.Client.transform(Client.java:160)
at sun.instrument.TransformerManager.transform(TransformerManager.java:169)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:365)
at java.util.HashMap.entrySet0(HashMap.java:945)
at java.util.HashMap.entrySet(HashMap.java:940)
at java.lang.Class.initAnnotationsIfNecessary(Class.java:3078)
- locked <0x000000060635c900> (a java.lang.Class for java.util.concurrent.ThreadPoolExecutor)
at java.lang.Class.getAnnotation(Class.java:3029)
at com.sun.btrace.runtime.ClassFilter.isCandidate(ClassFilter.java:95)
at com.sun.btrace.agent.Client.isCandidate(Client.java:321)
at com.sun.btrace.agent.Main$4.run(Main.java:417)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
"main":
at com.sun.btrace.runtime.ClassFilter.isCandidate(ClassFilter.java:158)
at com.sun.btrace.runtime.ClassFilter.isCandidate(ClassFilter.java:145)
at com.sun.btrace.agent.Client.transform(Client.java:160)
at sun.instrument.TransformerManager.transform(TransformerManager.java:169)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:365)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
- locked <0x00000007590e6b48> (a sun.misc.Launcher$AppClassLoader)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
- locked <0x00000007590e6b48> (a sun.misc.Launcher$AppClassLoader)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Found 1 deadlock.
问题:可以看到thread-1对某个对象加了锁,但是这个对象显示的是<unknown>
, 请问如何确定这个是什么锁?
PS:发现出现问题的代码行是创建了一个对象,该对象中有一个行代码创建了一个匿名类,并对其进行了实例化,如果将匿名类改成普通类,则死锁消失,不知道JVM在处理匿名类初始化的时候是否有特殊处理?
private class CheckingVisitor extends ClassVisitor {
private boolean isInterface;
private boolean isCandidate;
private AnnotationVisitor nullAnnotationVisitor = new AnnotationVisitor(Opcodes.ASM4) {};
从你的trace上看是pool-1-thread-1 锁住了ThreadPoolExecutor(0x000000060635c900), 并且想占有 AppClassLoader(0x00000007590e6b48); 然后main锁住了AppClassLoader(0x00000007590e6b48), 想占有(0x000000060644f890)这是个被pool-1-thread-1占有的, 总体来说就是main在执行AppClassLoader的public synchronized java.lang.Class loadClass(java.lang.String s, boolean b) 这个synchronized的方法之后想调用 ClassFilter的CheckingVisitor cv = new CheckingVisitor()最后卡在Class的synchronized void initAnnotationsIfNecessary()进不去; 然后pool-1-thread-1在执行Class的 synchronized void initAnnotationsIfNecessary();锁住之后最后想调用ClassFilter的CheckingVisitor cv = new CheckingVisitor(); 然后卡在loadClass进不去;死锁发生在loadClass和initAnnotationsIfNecessary两个syncronized方法的互相调用上。 所以如果不是匿名类而是普通类的话就不会出现死锁了。有点晕