<p dir="auto"><strong><a href="https://jira.spring.io/secure/ViewProfile.jspa?name=ggn" rel="nofollow">ggn</a></strong> opened <strong><a href="https://jira.spring.io/browse/SPR-9190?redirect=false" rel="nofollow">SPR-9190</a></strong> and commented</p>
<p dir="auto">We have been facing numerous threads being blocked waiting for getSingleton method on DefaultSingletonBeanRegistry class and because of this our servers have been hanging which requires a restart whenever it happens.</p>
<p dir="auto">We have been collecting threaddumps whenever this happens and we are consistently seeing as below for hundreds of threads:</p>
<p dir="auto">"catalina-exec-567" daemon prio=10 tid=0x00002aaabc5eb000 nid=0xc54 waiting for monitor entry [0x00002aaaf6320000]<br>
java.lang.Thread.State: BLOCKED (on object monitor)<br>
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)</p>
<ul dir="auto">
<li>waiting to lock &lt;0x00000005a2a58f70&gt; (a java.util.concurrent.ConcurrentHashMap)<br>
at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:466)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:319)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:298)<br>
at org.springframework.context.support.AbstractApplicationContext.getBeanNamesForType(AbstractApplicationContext.java:1132)</li>
</ul>
<p dir="auto">java.lang.Thread.State: BLOCKED (on object monitor)<br>
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)</p>
<ul dir="auto">
<li>waiting to lock &lt;0x00000005a2a58f70&gt; (a java.util.concurrent.ConcurrentHashMap)<br>
at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:466)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:319)<br>
at org.springframework.beans.factory.BeanFactoryUtils.beanNamesForTypeIncludingAncestors(BeanFactoryUtils.java:185)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:833)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:790)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:707)<br>
at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.autowireResource(CommonAnnotationBeanPostProcessor.java:433)<br>
at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.getResource(CommonAnnotationBeanPostProcessor.java:411)<br>
at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor$ResourceElement.getResourceToInject(CommonAnnotationBeanPostProcessor.java:543)<br>
at org.springframework.beans.factory.annotation.InjectionMetadata$InjectedElement.inject(InjectionMetadata.java:147)<br>
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:84)<br>
at org.springframework.context.annotation.CommonAnnotationBeanPostProcessor.postProcessPropertyValues(CommonAnnotationBeanPostProcessor.java:298)<br>
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1074)<br>
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)<br>
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)<br>
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:312)<br>
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)<br>
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1087)</li>
</ul>
<p dir="auto">I have already commented requesting for help on the related problem <a href="https://jira.springsource.org/browse/SPR-7949" rel="nofollow">https://jira.springsource.org/browse/SPR-7949</a> but haven't gotten any response.</p>
<p dir="auto"><a href="https://jira.springsource.org/browse/SPR-7949" rel="nofollow">https://jira.springsource.org/browse/SPR-7949</a> has some information about some patch but i haven't seen the patch being applied to 3.0.6 or 3.0.7 or 3.1.</p>
<p dir="auto">Please help with this issue as this is causing some major issue with our application.</p>
<p dir="auto">Thanks for help.<br>
ggn</p>
<hr>
<p dir="auto"><strong>Affects:</strong> 3.0.6</p>
<p dir="auto"><strong>Reference URL:</strong> <a href="https://jira.springsource.org/browse/SPR-7949" rel="nofollow">https://jira.springsource.org/browse/SPR-7949</a></p>
<p dir="auto"><strong>Issue Links:</strong></p>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="398110075" data-permission-text="Title is private" data-url="https://github.com/spring-projects/spring-framework/issues/12604" data-hovercard-type="issue" data-hovercard-url="/spring-projects/spring-framework/issues/12604/hovercard" href="https://github.com/spring-projects/spring-framework/issues/12604">#12604</a> Improve performance of DefaultSingletonBeanRegistry.getSingletonNames(..) for high-concurrent-load applications (<em><strong>"duplicates"</strong></em>)</li>
</ul>
<p dir="auto"><strong><a href="https://jira.spring.io/secure/ViewProfile.jspa?name=mck" rel="nofollow">mck</a></strong> opened <strong><a href="https://jira.spring.io/browse/SPR-7949?redirect=false" rel="nofollow">SPR-7949</a></strong> and commented</p>
<p dir="auto">Under high-concurrent load situation, in an application using many prototype beans, contention occurs in DefaultSingletonBeanRegistry.getSingletonNames(..)</p>
<p dir="auto">This method has uses singletonObjects as a mutex for all map operations.<br>
Using a ReadWriteLock this contention could be removed, as in our case 99% of the calls here are read-only.</p>
<p dir="auto">We see hundreds of thread dumps like in a single jvm:</p>
<p dir="auto">java.lang.Thread.State: BLOCKED (on object monitor)<br>
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)</p>
<ul dir="auto">
<li>waiting to lock &lt;0x00007fb8519b11b0&gt; (a java.util.concurrent.ConcurrentHashMap)<br>
at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:452)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:317)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:296)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:250)<br>
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1083)</li>
</ul>
<p dir="auto">java.lang.Thread.State: BLOCKED (on object monitor)<br>
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingletonNames(DefaultSingletonBeanRegistry.java:277)</p>
<ul dir="auto">
<li>waiting to lock &lt;0x00007fb856ab64d8&gt; (a java.util.concurrent.ConcurrentHashMap)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:352)<br>
at org.springframework.beans.factory.BeanFactoryUtils.beanNamesForTypeIncludingAncestors(BeanFactoryUtils.java:185)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:829)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:744)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:703)<br>
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.resolvedCachedArgument(AutowiredAnnotationBeanPostProcessor.java:436)<br>
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.access$0(AutowiredAnnotationBeanPostProcessor.java:432)<br>
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:469)<br>
at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:84)<br>
at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:283)<br>
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1074)<br>
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireBean(AbstractAutowireCapableBeanFactory.java:295)</li>
</ul>
<p dir="auto">java.lang.Thread.State: BLOCKED (on object monitor)<br>
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:180)</p>
<ul dir="auto">
<li>waiting to lock &lt;0x00007fb8519b11b0&gt; (a java.util.concurrent.ConcurrentHashMap)<br>
at org.springframework.beans.factory.support.AbstractBeanFactory.isTypeMatch(AbstractBeanFactory.java:452)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:317)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:296)<br>
at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBean(DefaultListableBeanFactory.java:250)<br>
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1083)</li>
</ul>
<hr>
<p dir="auto"><strong>Affects:</strong> 3.0.4, 3.0.5</p>
<p dir="auto"><strong>Attachments:</strong></p>
<ul dir="auto">
<li><a href="https://jira.spring.io/secure/attachment/17615/SPR-7949.patch" rel="nofollow">SPR-7949.patch</a> (<em>15.19 kB</em>)</li>
</ul>
<p dir="auto"><strong>Issue Links:</strong></p>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="398103107" data-permission-text="Title is private" data-url="https://github.com/spring-projects/spring-framework/issues/11536" data-hovercard-type="issue" data-hovercard-url="/spring-projects/spring-framework/issues/11536/hovercard" href="https://github.com/spring-projects/spring-framework/issues/11536">#11536</a> Cache by-type lookups in DefaultListableBeanFactory (<em><strong>"duplicates"</strong></em>)</li>
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="398117731" data-permission-text="Title is private" data-url="https://github.com/spring-projects/spring-framework/issues/13828" data-hovercard-type="issue" data-hovercard-url="/spring-projects/spring-framework/issues/13828/hovercard" href="https://github.com/spring-projects/spring-framework/issues/13828">#13828</a> Threads blocking on getSingleton method on DefaultSingletonBeanRegistry class  (<em><strong>"is duplicated by"</strong></em>)</li>
</ul>
<p dir="auto">17 votes, 16 watchers</p>