title: Classpath problems while Instrumenting Springboot application
body: <p>I have a springboot application which I'm trying to instrument using bytebuddy. I'm running into classpath issues which I'm not able to understand. </p>
  
  <p>Firstly, the following is other literature on this:</p>
  
  <p><a href="https://github.com/raphw/byte-buddy/issues/473" rel="nofollow noreferrer">https://github.com/raphw/byte-buddy/issues/473</a></p>
  
  <p><a href="https://github.com/raphw/byte-buddy/issues/87" rel="nofollow noreferrer">https://github.com/raphw/byte-buddy/issues/87</a></p>
  
  <p><a href="https://stackoverflow.com/questions/34523361/unable-to-instrument-apache-httpclient-using-javaagent-for-spring-boot-uber-jar">Unable to instrument apache httpclient using javaagent for spring boot uber jar application</a></p>
  
  <p><a href="https://github.com/raphw/byte-buddy/issues/109" rel="nofollow noreferrer">https://github.com/raphw/byte-buddy/issues/109</a></p>
  
  <p><a href="https://github.com/raphw/byte-buddy/issues/473" rel="nofollow noreferrer">https://github.com/raphw/byte-buddy/issues/473</a></p>
  
  <p><a href="https://github.com/raphw/byte-buddy/issues/489" rel="nofollow noreferrer">https://github.com/raphw/byte-buddy/issues/489</a></p>
  
  <p><a href="https://github.com/spring-projects/spring-boot/issues/4868" rel="nofollow noreferrer">https://github.com/spring-projects/spring-boot/issues/4868</a></p>
  
  <p><a href="https://github.com/alibaba/transmittable-thread-local/issues/161" rel="nofollow noreferrer">https://github.com/alibaba/transmittable-thread-local/issues/161</a></p>
  
  <p>Problem is that Spring-boot bundles the application into one uber-jar, which contains other jars inside it</p>
  
  <p><a href="https://i.stack.imgur.com/fe0ze.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fe0ze.jpg" alt="spring-boot-jar"></a></p>
  
  <p>A thing to note here is, that if I run the application using IntelliJ, it doesn't use the uber-jar and runs via main class with a bunch of jars as classpath arguments. </p>
  
  <p>Due to this difference, When running via uber-jar(<code>java -jar target/demo-0.0.1-SNAPSHOT.jar</code>), some classes are not available at the time the Agent runs. The classes are loadable at the time of application main, as spring-boot uses its own classloader, which is created at some time b/w agent and application main methods. </p>
  
  <p>I'll describe the behaviour at various points of time below:</p>
  
  <h2>At time of PreMain of Agent</h2>
  
  <ul>
  <li><code>Thread.currentThread().getContextClassLoader() = Launcher$AppClassLoader</code></li>
  <li><code>Agent.class.classLoader = Launcher$AppClassLoader</code></li>
  <li><code>Class.forName("org.springframework.web.servlet.HandlerAdapter") =&gt; ClassNotFoundException</code></li>
  <li><code>Class.forName("javax.servlet.http.HttpServletRequest") =&gt; ClassNotFoundException</code></li>
  </ul>
  
  <p>Both spring and javax classes are not loaded as they are not directly in the classpath as per the App Classloader. It's part of an inner jar, something like <code>app.jar!/BOOT-INF/lib/some.jar</code>
  App Classloader won't be able to load this.</p>
  
  <h2>At time of Main of Application Class</h2>
  
  <ul>
  <li><code>Thread.currentThread().getContextClassLoader() =</code> <code>org.springframework.boot.loader.LaunchedURLClassLoader</code></li>
  <li><code>DemoApplication.class.classloader = same as above</code></li>
  <li><code>Class.forName("org.springframework.web.servlet.HandlerAdapter") =&gt; loads successfully</code></li>
  <li>Same for javax class, loads successfully. </li>
  </ul>
  
  <p>After loading, <code>Class.forName("javax.servlet.http.HttpServletRequest").classLoader</code> is also the same <code>LaunchedURLClassLoader</code>.</p>
  
  <h2>At the time of <code>builder.visit</code> call (when bytebuddy is modifying the class definitions)</h2>
  
  <h3>Code:</h3>
  
  <p>I have two classes, <code>SpringBootInterceptor</code> (which contains the intercept method) and <code>SpringBootInterceptorOne</code> (which contains the entry and exit method)</p>
  
  <pre class="lang-java prettyprint-override"><code>  @Override
    public AgentBuilder intercept(AgentBuilder agentBuilder) {
      return agentBuilder
        .type(isSubTypeOf(HandlerAdapter.class))
        .transform((builder, typeDescription, classLoader, module) -&gt;
          builder.visit(Advice.to(SpringBootInterceptorOne.class, SpringBootInterceptorOne.class).on(named("handle").and(isMethod())))
        );
    }
  
  </code></pre>
  
  <p>The function <code>intercept()</code> is called in the premain path. </p>
  
  <p>The function <code>visit</code> is called when the type is actually attempted to load. At that point bytebuddy tris to intercept those methods and modify bytecode. </p>
  
  <p>In the current version of this code, the function <code>intercept()</code> throws <code>ClassNotFoundException</code> and interception doesn't happen. </p>
  
  <pre class="lang-java prettyprint-override"><code>Caused by: java.lang.NoClassDefFoundError: org/springframework/web/servlet/HandlerAdapter
  </code></pre>
  
  <p>So, I tried to change the code, to the following:</p>
  
  <pre class="lang-java prettyprint-override"><code>  @Override
    public AgentBuilder intercept(AgentBuilder agentBuilder) {
      return agentBuilder
  //      .type(isSubTypeOf(HandlerAdapter.class))
        .type(not(isInterface())
          .and(safeHasSuperType(named("org.springframework.web.servlet.HandlerAdapter"))))
        .transform((builder, typeDescription, classLoader, module) -&gt;
          builder.visit(Advice.to(SpringBootInterceptorOne.class, SpringBootInterceptorOne.class).on(named("handle").and(isMethod())))
        );
    }
  </code></pre>
  
  <p>Here, basically loading the class lazily.
  Helper functions from datadog: </p>
  
  <p><a href="https://github.com/DataDog/dd-trace-java/blob/master/dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/ByteBuddyElementMatchers.java" rel="nofollow noreferrer">https://github.com/DataDog/dd-trace-java/blob/master/dd-java-agent/agent-tooling/src/main/java/datadog/trace/agent/tooling/ByteBuddyElementMatchers.java</a></p>
  
  <p>After doing this, the <code>intercept</code> function doesn't fail, but the <code>visit()</code> function behaves strangely:</p>
  
  <pre class="lang-java prettyprint-override"><code>java.lang.IllegalStateException: Could not locate class file for my.package.name.SpringBootInterceptorOne
      at net.bytebuddy.dynamic.ClassFileLocator$Resolution$Illegal.resolve(ClassFileLocator.java:118)
      at net.bytebuddy.asm.Advice.to(Advice.java:431)
      at net.bytebuddy.asm.Advice.to(Advice.java:400)
      at net.bytebuddy.asm.Advice.to(Advice.java:375)
      at net.bytebuddy.asm.Advice.to(Advice.java:361)
      at my.package.name.SpringBootInterceptor.lambda$intercept$0(SpringBootInterceptor.java:30)
      at net.bytebuddy.agent.builder.AgentBuilder$Transformer$Compound.transform(AgentBuilder.java:2612)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$Transformation$Simple$Resolution.apply(AgentBuilder.java:10117)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10494)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10457)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1500(AgentBuilder.java:10223)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:10833)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$LegacyVmDispatcher.run(AgentBuilder.java:10780)
      at java.security.AccessController.doPrivileged(Native Method)
      at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10380)
      at sun.instrument.TransformerManager.transform(TransformerManager.java:188)
      at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:428)
      at java.lang.ClassLoader.defineClass1(Native Method)
      at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
      at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
      at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
      at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
      at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:92)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
      at java.lang.Class.getDeclaredMethods0(Native Method)
      at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
      at java.lang.Class.getDeclaredMethods(Class.java:1975)
      at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:463)
      at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:358)
      at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:414)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.lambda$getTypeForFactoryMethod$2(AbstractAutowireCapableBeanFactory.java:743)
      at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1688)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:742)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:681)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:649)
      at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1604)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:520)
      at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:491)
      at org.springframework.boot.autoconfigure.condition.OnBeanCondition.collectBeanNamesForType(OnBeanCondition.java:230)
      at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getBeanNamesForType(OnBeanCondition.java:223)
      at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getBeanNamesForType(OnBeanCondition.java:213)
      at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getMatchingBeans(OnBeanCondition.java:167)
      at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getMatchOutcome(OnBeanCondition.java:142)
      at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:47)
      at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108)
      at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForBeanMethod(ConfigurationClassBeanDefinitionReader.java:184)
      at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:144)
      at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:120)
      at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:331)
      at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:236)
      at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:275)
      at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:95)
      at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:706)
      at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:532)
      at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
      at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
      at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
      at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
      at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
      at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
      at com.example.demo.DemoApplication.main(DemoApplication.java:10)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
      at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
      at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
      at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
  
  </code></pre>
  
  <p>At this point, </p>
  
  <ul>
  <li><code>Thread.currentThread().getContextClassLoader() =&gt; LaunchedURLClassLoader</code></li>
  <li><code>SpringBootInterceptor.class.getClassLoader() = null</code></li>
  <li>Typing <code>SpringBootInterceptor.class</code> in the debugger window doesn't fail, but</li>
  <li><code>Class.forName("my.package.name.SpringBootInterceptorOne")</code> => NoClassDefFound</li>
  </ul>
  
  <p>Next, I tried the following:</p>
  
  <pre class="lang-java prettyprint-override"><code>  @Override
    public AgentBuilder intercept(AgentBuilder agentBuilder) {
      ClassLoader cl = Thread.currentThread().getContextClassLoader();
      return agentBuilder
  //      .type(isSubTypeOf(HandlerAdapter.class))
        .type(not(isInterface())
          .and(safeHasSuperType(named("org.springframework.web.servlet.HandlerAdapter"))))
        .transform((builder, typeDescription, classLoader, module) -&gt; {
            try {
              Class&lt;?&gt; claz = Thread.currentThread().getContextClassLoader().loadClass("my.package.name.SpringBootInterceptorOne");
              return builder.visit(Advice.to(claz, claz,
                new ClassFileLocator.Compound(ClassFileLocator.ForClassLoader.of(cl),
                ClassFileLocator.ForClassLoader.of(Thread.currentThread().getContextClassLoader())
                )).on(named("handle").and(isMethod())));
            } catch (ClassNotFoundException e) {
              e.printStackTrace();
              return builder;
            }
          }
        );
    }
  
  </code></pre>
  
  <p>This causes the interceptor to be "installed". The entry and exit methods are called okay. But casting the arguments to proper types gives error for the types of the arguments</p>
  
  <pre class="lang-java prettyprint-override"><code>exit spring boot
  [Ljava.lang.Object;@c83066a
  java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest
      at my.package.name.SpringBootInterceptorOne.getEndpoint(SpringBootInterceptorOne.java:36)
      at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
      at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
      at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
      at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
      at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
      at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
      at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
      at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
      at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860)
      at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598)
      at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      at java.lang.Thread.run(Thread.java:748)
  
  </code></pre>
  
  <p>Here is SpringBootInterceptorOne for reference:</p>
  
  <pre class="lang-java prettyprint-override"><code>  @Advice.OnMethodExit
    static void exit(@Advice.Origin final Executable executable,
                     @Advice.This Object handlerAdapter,
                     @Advice.AllArguments Object[] arguments,
                     @Advice.Enter final TimerContext ctx) {
  //      final long duration = ctx.getDuration();
      System.out.println("exit spring boot");
      String e = getEndpoint(arguments);
      System.out.println(e);
    }
  
    public static String getEndpoint(Object[] arguments) {
      System.out.println(arguments);
      HttpServletRequest request = (HttpServletRequest) arguments[0];
      HandlerMethod  handler = (HandlerMethod) arguments[2];
      return "ad";
    }
  </code></pre>
  
  <p>UPDATE:</p>
  
  <p>As per the advice offered in an answer, I tried the following:</p>
  
  <pre class="lang-java prettyprint-override"><code>  @Override
    public AgentBuilder intercept(AgentBuilder agentBuilder) {
      return agentBuilder
  //      .type(isSubTypeOf(HandlerAdapter.class))
        .type(not(isInterface())
          .and(safeHasSuperType(named("org.springframework.web.servlet.HandlerAdapter"))))
        .transform(new AgentBuilder.Transformer.ForAdvice()
          .include(SpringBootInterceptorOne.class.getClassLoader())
          .include(getClass().getClassLoader())
          .advice(named("handle"), SpringBootInterceptorOne.class.getName())
        );
    }
  
  </code></pre>
  
  <p>It again throws Error:</p>
  
  <pre class="lang-java prettyprint-override"><code>java.lang.NoClassDefFoundError: javax/servlet/http/HttpServletRequest
      at io.signoz.interceptors.http.server.springboot.SpringBootInterceptorOne.getEndpoint(SpringBootInterceptorOne.java:37) ~[na:na]
      at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.3.RELEASE.jar!/:5.2.3.RELEASE]
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:860) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1598) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_242]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_242]
      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.30.jar!/:9.0.30]
      at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242]
  
  </code></pre>