Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Menu filtering fails because of database locks #9282

Open
fdlk opened this issue Jul 22, 2021 · 2 comments
Open

Menu filtering fails because of database locks #9282

fdlk opened this issue Jul 22, 2021 · 2 comments
Labels
9.0 bug

Comments

@fdlk
Copy link
Contributor

fdlk commented Jul 22, 2021 •
edited

How to Reproduce

Don't know, we were importing a large file for test case IMPORTASYNC001 (Asynchronous upload)

Expected behavior

Users can concurrently use MOLGENIS

Observed behavior

Worker threads are hogged, all waiting for the database

"http-nio-8080-exec-20" #1267 daemon prio=5 os_prio=0 cpu=15.28ms elapsed=37.76s tid=0x00007f0668015000 nid=0x4fa runnable  [0x00007f0651df3000] 
   java.lang.Thread.State: RUNNABLE 
	at java.net.SocketInputStream.socketRead0(java.base@11.0.10/Native Method) 
	at java.net.SocketInputStream.socketRead(java.base@11.0.10/SocketInputStream.java:115) 
	at java.net.SocketInputStream.read(java.base@11.0.10/SocketInputStream.java:168) 
	at java.net.SocketInputStream.read(java.base@11.0.10/SocketInputStream.java:140) 
	at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161) 
	at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:128) 
	at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:113) 
	at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73) 
	at org.postgresql.core.PGStream.receiveChar(PGStream.java:370) 
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2044) 
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:313) 
	- locked <0x000000009987a268> (a org.postgresql.core.v3.QueryExecutorImpl) 
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448) 
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369) 
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:159) 
	at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:109) 
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) 
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) 
	at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:678) 
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:617) 
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:669) 
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700) 
	at org.springframework.security.acls.jdbc.BasicLookupStrategy.lookupObjectIdentities(BasicLookupStrategy.java:381) 
	at org.springframework.security.acls.jdbc.BasicLookupStrategy.readAclsById(BasicLookupStrategy.java:336) 
	at org.springframework.security.acls.jdbc.JdbcAclService.readAclsById(JdbcAclService.java:129) 
	at org.molgenis.security.acl.TransactionalJdbcMutableAclService.readAclsById(TransactionalJdbcMutableAclService.java:117) 
	at org.springframework.security.acls.jdbc.JdbcAclService.readAclById(JdbcAclService.java:111) 
	at org.molgenis.security.acl.TransactionalJdbcMutableAclService.readAclById(TransactionalJdbcMutableAclService.java:99) 
	at org.molgenis.security.acl.TransactionalJdbcMutableAclService$$FastClassBySpringCGLIB$$f78d44cb.invoke(<generated>) 
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.transaction.interceptor.TransactionInterceptor$$Lambda$1152/0x0000000840b01040.proceedWithInvocation(Unknown Source) 
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) 
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) 
	at org.molgenis.security.acl.TransactionalJdbcMutableAclService$$EnhancerBySpringCGLIB$$f195ec4b.readAclById(<generated>) 
	at org.springframework.security.acls.AclPermissionEvaluator.checkPermission(AclPermissionEvaluator.java:103) 
	at org.springframework.security.acls.AclPermissionEvaluator.hasPermission(AclPermissionEvaluator.java:85) 
	at org.molgenis.security.permission.UserPermissionEvaluatorImpl.hasPermission(UserPermissionEvaluatorImpl.java:39) 
	at org.molgenis.web.menu.MenuReaderServiceImpl.userHasViewPermission(MenuReaderServiceImpl.java:32) 
	at org.molgenis.web.menu.MenuReaderServiceImpl$$Lambda$1772/0x0000000840dbc440.test(Unknown Source) 
	at java.util.stream.ReferencePipeline$2$1.accept(java.base@11.0.10/ReferencePipeline.java:176) 
	at java.util.stream.ReferencePipeline$3$1.accept(java.base@11.0.10/ReferencePipeline.java:195) 
	at java.util.stream.ReferencePipeline$2$1.accept(java.base@11.0.10/ReferencePipeline.java:177) 
	at java.util.stream.ReferencePipeline$3$1.accept(java.base@11.0.10/ReferencePipeline.java:195) 
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@11.0.10/ArrayList.java:1655) 
	at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.10/AbstractPipeline.java:484) 
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.10/AbstractPipeline.java:474) 
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@11.0.10/ReduceOps.java:913) 
	at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.10/AbstractPipeline.java:234) 
	at java.util.stream.ReferencePipeline.collect(java.base@11.0.10/ReferencePipeline.java:578) 
	at org.molgenis.web.menu.model.Menu.filter(Menu.java:30) 
	at org.molgenis.web.menu.model.Menu.lambda$filter$0(Menu.java:26) 
	at org.molgenis.web.menu.model.Menu$$Lambda$1773/0x0000000840dbc840.apply(Unknown Source) 
	at java.util.stream.ReferencePipeline$3$1.accept(java.base@11.0.10/ReferencePipeline.java:195) 
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@11.0.10/ArrayList.java:1655) 
	at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.10/AbstractPipeline.java:484) 
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.10/AbstractPipeline.java:474) 
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@11.0.10/ReduceOps.java:913) 
	at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.10/AbstractPipeline.java:234) 
	at java.util.stream.ReferencePipeline.collect(java.base@11.0.10/ReferencePipeline.java:578) 
	at org.molgenis.web.menu.model.Menu.filter(Menu.java:30) 
	at org.molgenis.web.menu.MenuReaderServiceImpl.lambda$getMenu$1(MenuReaderServiceImpl.java:42) 
	at org.molgenis.web.menu.MenuReaderServiceImpl$$Lambda$1771/0x0000000840dbc040.apply(Unknown Source) 
	at java.util.Optional.flatMap(java.base@11.0.10/Optional.java:294) 
	at org.molgenis.web.menu.MenuReaderServiceImpl.getMenu(MenuReaderServiceImpl.java:42) 
	at org.molgenis.core.ui.MolgenisMenuController.forwardDefaultMenuDefaultPlugin(MolgenisMenuController.java:83) 
	at org.molgenis.core.ui.MolgenisMenuController$$FastClassBySpringCGLIB$$206c924a.invoke(<generated>) 
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.transaction.interceptor.TransactionInterceptor$$Lambda$1152/0x0000000840b01040.proceedWithInvocation(Unknown Source) 
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:366) 
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) 
	at org.molgenis.core.ui.MolgenisMenuController$$EnhancerBySpringCGLIB$$f1d4bc28.forwardDefaultMenuDefaultPlugin(<generated>) 
	at jdk.internal.reflect.GeneratedMethodAccessor216.invoke(Unknown Source) 
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.10/DelegatingMethodAccessorImpl.java:43) 
	at java.lang.reflect.Method.invoke(java.base@11.0.10/Method.java:566) 
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) 
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) 
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105) 
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879) 
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) 
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) 
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) 
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) 
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) 
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) 
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:626) 
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) 
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:710) 
	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:457) 
	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:384) 
	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312) 
	at org.springframework.security.web.header.HeaderWriterFilter$HeaderWriterRequestDispatcher.forward(HeaderWriterFilter.java:172) 
	at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:171) 
	at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:316) 
	at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1373) 
	at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1118) 
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1057) 
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) 
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) 
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) 
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:626) 
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) 
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:106) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320) 
	at org.molgenis.security.twofactor.auth.TwoFactorAuthenticationFilter.doFilterInternal(TwoFactorAuthenticationFilter.java:54) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.molgenis.security.MolgenisChangePasswordFilter.doFilter(MolgenisChangePasswordFilter.java:52) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126) 
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.molgenis.security.token.TokenAuthenticationFilter.doFilter(TokenAuthenticationFilter.java:52) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:155) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.oauth2.client.web.OAuth2AuthorizationRequestRedirectFilter.doFilterInternal(OAuth2AuthorizationRequestRedirectFilter.java:160) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92) 
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334) 
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215) 
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178) 
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) 
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.springframework.web.filter.ForwardedHeaderFilter.doFilterInternal(ForwardedHeaderFilter.java:158) 
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.molgenis.core.ui.browserdetection.BrowserDetectionFilter.doFilter(BrowserDetectionFilter.java:52) 
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189) 
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162) 
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) 
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) 
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542) 
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) 
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) 
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687) 
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) 
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357) 
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374) 
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) 
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:893) 
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1707) 
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) 
	- locked <0x0000000080c22d28> (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper) 
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.10/ThreadPoolExecutor.java:1128) 
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.10/ThreadPoolExecutor.java:628) 
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
	at java.lang.Thread.run(java.base@11.0.10/Thread.java:834)
@fdlk fdlk changed the title Menu filtering fails because of deadlock Menu filtering fails because of database locks Jul 22, 2021
@fdlk
Copy link
Contributor Author

fdlk commented Jul 22, 2021

The commit of the import takes a looong time and is busy retrieving metadata.

"pool-5-thread-1" #112 prio=5 os_prio=0 cpu=4766.81ms elapsed=539.78s tid=0x00007f9ae4050000 nid=0x77 runnable  [0x00007f9b367ca000] 
   java.lang.Thread.State: RUNNABLE 
	at java.net.SocketInputStream.socketRead0(java.base@11.0.10/Native Method) 
	at java.net.SocketInputStream.socketRead(java.base@11.0.10/SocketInputStream.java:115) 
	at java.net.SocketInputStream.read(java.base@11.0.10/SocketInputStream.java:168) 
	at java.net.SocketInputStream.read(java.base@11.0.10/SocketInputStream.java:140) 
	at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:161) 
	at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:128) 
	at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:113) 
	at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:223) 
	at org.postgresql.core.PGStream.receive(PGStream.java:529) 
	at org.postgresql.core.PGStream.receiveTupleV3(PGStream.java:489) 
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2237) 
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:313) 
	- locked <0x00000000878f63a8> (a org.postgresql.core.v3.QueryExecutorImpl) 
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448) 
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369) 
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:159) 
	at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:109) 
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52) 
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java) 
	at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:678) 
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:617) 
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:669) 
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:700) 
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:712) 
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:763) 
	at org.molgenis.data.postgresql.PostgreSqlRepository$1.getBatch(PostgreSqlRepository.java:464) 
	at org.molgenis.data.support.BatchingQueryResult.getBatch(BatchingQueryResult.java:29) 
	at org.molgenis.data.support.BatchingQueryResult.getBatch(BatchingQueryResult.java:13) 
	at org.molgenis.data.util.BatchingIterable$1.nextBatch(BatchingIterable.java:106) 
	at org.molgenis.data.util.BatchingIterable$1.hasNext(BatchingIterable.java:48) 
	at java.util.Spliterators$IteratorSpliterator.tryAdvance(java.base@11.0.10/Spliterators.java:1811) 
	at java.util.Spliterators$1Adapter.hasNext(java.base@11.0.10/Spliterators.java:681) 
	at com.google.common.collect.Maps.uniqueIndex(Maps.java:1273) 
	at org.molgenis.data.support.AbstractRepository.findAllBatched(AbstractRepository.java:109) 
	at org.molgenis.data.support.AbstractRepository.lambda$findAll$1(AbstractRepository.java:100) 
	at org.molgenis.data.support.AbstractRepository$$Lambda$1323/0x0000000840bc3840.apply(Unknown Source) 
	at java.util.stream.ReferencePipeline$7$1.accept(java.base@11.0.10/ReferencePipeline.java:271) 
	at java.util.Iterator.forEachRemaining(java.base@11.0.10/Iterator.java:133) 
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(java.base@11.0.10/Spliterators.java:1801) 
	at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.10/AbstractPipeline.java:484) 
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.10/AbstractPipeline.java:474) 
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@11.0.10/ForEachOps.java:150) 
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@11.0.10/ForEachOps.java:173) 
	at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.10/AbstractPipeline.java:234) 
	at java.util.stream.ReferencePipeline.forEach(java.base@11.0.10/ReferencePipeline.java:497) 
	at org.molgenis.data.cache.l1.L1CacheRepositoryDecorator.findAllWithCache(L1CacheRepositoryDecorator.java:146) 
	at org.molgenis.data.cache.l1.L1CacheRepositoryDecorator.lambda$findAllWithCache$0(L1CacheRepositoryDecorator.java:107) 
	at org.molgenis.data.cache.l1.L1CacheRepositoryDecorator$$Lambda$1313/0x0000000840bc1040.apply(Unknown Source) 
	at java.util.stream.ReferencePipeline$3$1.accept(java.base@11.0.10/ReferencePipeline.java:195) 
	at java.util.Spliterators$IteratorSpliterator.tryAdvance(java.base@11.0.10/Spliterators.java:1812) 
	at java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(java.base@11.0.10/StreamSpliterators.java:294) 
	at java.util.stream.StreamSpliterators$WrappingSpliterator$$Lambda$1219/0x0000000840b69840.getAsBoolean(java.base@11.0.10/Unknown Source) 
	at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(java.base@11.0.10/StreamSpliterators.java:206) 
	at java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(java.base@11.0.10/StreamSpliterators.java:161) 
	at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(java.base@11.0.10/StreamSpliterators.java:300) 
	at java.util.Spliterators$1Adapter.hasNext(java.base@11.0.10/Spliterators.java:681) 
	at com.google.common.collect.Iterators$4.hasNext(Iterators.java:600) 
	at java.util.Iterator.forEachRemaining(java.base@11.0.10/Iterator.java:132) 
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(java.base@11.0.10/Spliterators.java:1801) 
	at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.10/AbstractPipeline.java:484) 
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.10/AbstractPipeline.java:474) 
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(java.base@11.0.10/ReduceOps.java:913) 
	at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.10/AbstractPipeline.java:234) 
	at java.util.stream.ReferencePipeline.collect(java.base@11.0.10/ReferencePipeline.java:578) 
	at org.molgenis.data.EntityManagerImpl.resolveReferences(EntityManagerImpl.java:196) 
	at org.molgenis.data.EntityManagerImpl.lambda$resolveReferences$1(EntityManagerImpl.java:143) 
	at org.molgenis.data.EntityManagerImpl$$Lambda$1440/0x0000000840bfb040.apply(Unknown Source) 
	at java.util.stream.ReferencePipeline$7$1.accept(java.base@11.0.10/ReferencePipeline.java:271) 
	at java.util.Iterator.forEachRemaining(java.base@11.0.10/Iterator.java:133) 
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(java.base@11.0.10/Spliterators.java:1801) 
	at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.10/AbstractPipeline.java:484) 
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.10/AbstractPipeline.java:474) 
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@11.0.10/ForEachOps.java:150) 
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@11.0.10/ForEachOps.java:173) 
	at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.10/AbstractPipeline.java:234) 
	at java.util.stream.ReferencePipeline.forEach(java.base@11.0.10/ReferencePipeline.java:497) 
	at org.molgenis.data.index.IndexActionRegisterServiceImpl.getEntityTypes(IndexActionRegisterServiceImpl.java:183) 
	at org.molgenis.data.index.IndexActionRegisterServiceImpl.createIndexDependencyModel(IndexActionRegisterServiceImpl.java:148) 
	at org.molgenis.data.index.IndexActionRegisterServiceImpl.storeIndexActions(IndexActionRegisterServiceImpl.java:124) 
	at org.molgenis.data.index.IndexActionRegisterServiceImpl$$FastClassBySpringCGLIB$$501b3016.invoke(<generated>) 
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) 
	at org.molgenis.security.core.runas.RunAsSystemAspect$$Lambda$1144/0x0000000840b0b840.run(Unknown Source) 
	at org.molgenis.security.core.runas.RunAsSystemAspect.runAsSystem(RunAsSystemAspect.java:49) 
	at org.molgenis.security.core.runas.RunAsSystemAspect.aroundAdvice(RunAsSystemAspect.java:28) 
	at jdk.internal.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) 
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.10/DelegatingMethodAccessorImpl.java:43) 
	at java.lang.reflect.Method.invoke(java.base@11.0.10/Method.java:566) 
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) 
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) 
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) 
	at org.molgenis.data.index.IndexActionRegisterServiceImpl$$EnhancerBySpringCGLIB$$76f73b23.storeIndexActions(<generated>) 
	at org.molgenis.data.index.transaction.IndexTransactionListener.commitTransaction(IndexTransactionListener.java:26) 
	at org.molgenis.data.postgresql.transaction.PostgreSqlTransactionManager.lambda$doCommit$1(PostgreSqlTransactionManager.java:110) 
	at org.molgenis.data.postgresql.transaction.PostgreSqlTransactionManager$$Lambda$1435/0x0000000840bf9c40.accept(Unknown Source) 
	at java.util.ArrayList.forEach(java.base@11.0.10/ArrayList.java:1541) 
	at org.molgenis.data.postgresql.transaction.PostgreSqlTransactionManager.doCommit(PostgreSqlTransactionManager.java:110) 
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743) 
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711) 
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:631) 
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:385) 
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) 
	at org.molgenis.data.importer.emx.ImportWriter$$EnhancerBySpringCGLIB$$8ba3fc50.doImport(<generated>) 
	at org.molgenis.data.importer.emx.EmxImportService.doImport(EmxImportService.java:100) 
	at org.molgenis.data.importer.emx.EmxImportService.doImport(EmxImportService.java:88) 
	at org.molgenis.data.importer.emx.EmxImportService$$FastClassBySpringCGLIB$$d53532ea.invoke(<generated>) 
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) 
	at org.molgenis.js.magma.WithJsMagmaScriptAspect$$Lambda$2101/0x0000000840fc0840.run(Unknown Source) 
	at org.molgenis.js.magma.WithJsMagmaScriptAspect.withJsMagmaScriptContext(WithJsMagmaScriptAspect.java:47) 
	at org.molgenis.js.magma.WithJsMagmaScriptAspect.aroundAdvice(WithJsMagmaScriptAspect.java:30) 
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.10/Native Method) 
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.10/NativeMethodAccessorImpl.java:62) 
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.10/DelegatingMethodAccessorImpl.java:43) 
	at java.lang.reflect.Method.invoke(java.base@11.0.10/Method.java:566) 
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) 
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) 
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) 
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) 
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) 
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) 
	at org.molgenis.data.importer.emx.EmxImportService$$EnhancerBySpringCGLIB$$6b4396d5.doImport(<generated>) 
	at org.molgenis.data.importer.ImportJob.run(ImportJob.java:66) 
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.10/ThreadPoolExecutor.java:1128) 
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.10/ThreadPoolExecutor.java:628) 
	at java.lang.Thread.run(java.base@11.0.10/Thread.java:834)

@fdlk
Copy link
Contributor Author

fdlk commented Aug 4, 2021

We think this is sort of the same as #8670

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
9.0 bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fdlk