title: NullPointerException in unit tests after migrating from groovy to kotlin gradle script
body: <p>I have no idea what's happening here, there shouldn't be a connection. Not one that manifests this specifically, anyways.</p>
  <p>After moving the buildscript of a major spring-boot project from groovy to kotlin-DSL, some unit tests are throwing NullPointerException. It's all the unit tests that deal with methods that have manual transaction management using a TransactionTemplate. This has not happened before the move, and by now I have compared the old and the new buildscript a dozen times over and am pretty confident that I didn't forget anything.</p>
  <p>Let's start out by comparing the two versions of the build scripts.</p>
  <p>Old groovy:</p>
  <pre><code>buildscript {
      ext {
          kotlinVersion = '1.4.10'
          springBootVersion = '2.3.3.RELEASE'
          awsSdkVersion = '1.11.381'
          springfoxVersion = '2.9.2'
          kotlintestVersion = '3.1.7'
      }
      repositories {
          mavenLocal()
          jcenter()
          mavenCentral()
      }
      dependencies {
          classpath &quot;org.springframework.boot:spring-boot-gradle-plugin:$springBootVersion&quot;
          classpath &quot;org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlinVersion&quot;
          classpath &quot;org.jetbrains.kotlin:kotlin-allopen:$kotlinVersion&quot;
          classpath &quot;org.jetbrains.kotlin:kotlin-noarg:$kotlinVersion&quot;
  
          // liquibase stuff
          classpath 'mysql:mysql-connector-java:5.1.44'
          classpath 'org.yaml:snakeyaml:1.19'
          classpath('org.liquibase:liquibase-gradle-plugin:1.2.3') {
              exclude group: &quot;org.liquibase&quot;, module: &quot;liquibase-core&quot;
          }
          classpath &quot;org.liquibase:liquibase-core:3.5.3&quot; // should be in sync with the version included in spring boot
      }
  }
  
  plugins {
      id 'net.researchgate.release' version '2.6.0'
  }
  
  apply plugin: 'kotlin'
  apply plugin: 'kotlin-spring'
  apply plugin: 'kotlin-jpa'
  apply plugin: 'org.springframework.boot'
  apply plugin: 'io.spring.dependency-management'
  apply plugin: 'jacoco'
  apply plugin: 'org.liquibase.gradle'
  apply plugin: 'idea'
  
  group = 'webcam.yellow.service'
  sourceCompatibility = 1.8
  
  compileKotlin {
      kotlinOptions.jvmTarget = &quot;1.8&quot;
  }
  compileTestKotlin {
      kotlinOptions.jvmTarget = &quot;1.8&quot;
  }
  
  jacocoTestReport {
      reports {
          xml.enabled = true
          html.enabled = true
      }
  }
  
  check.dependsOn jacocoTestReport
  
  repositories {
      mavenLocal()
      mavenCentral()
      jcenter()
      maven { url = &quot;http://clojars.org/repo/&quot; }
      maven {
          credentials {
              username &quot;${mavenUser}&quot;
              password &quot;${mavenPassword}&quot;
          }
          url = &quot;https://artifactory.yellow.webcam/artifactory/releases&quot;
      }
  
      maven {
          credentials {
              username &quot;${mavenUser}&quot;
              password &quot;${mavenPassword}&quot;
          }
          url = &quot;https://artifactory.yellow.webcam/artifactory/snapshots&quot;
      }
  
  }
  
  idea {
      module {
          // if you hate browsing Javadoc
          downloadJavadoc = false
          // and love reading sources :)
          downloadSources = true
      }
  }
  
  dependencies {
      // avisec libraries
      compile &quot;webcam.yellow.api:webcam-api:3.2.0&quot;
      compile &quot;webcam.yellow.authentication:messaging-authentication:1.5&quot;
  
      // Spring Boot
      compile 'org.springframework.boot:spring-boot-starter-actuator'
      compile 'org.springframework.boot:spring-boot-starter-data-jpa'
      compile 'org.springframework.boot:spring-boot-starter-web'
      compile 'org.springframework.boot:spring-boot-starter-security'
      compile 'org.springframework.boot:spring-boot-starter-activemq'
      compile 'org.springframework.boot:spring-boot-starter-validation'
  
      compile 'io.micrometer:micrometer-registry-influx'
  
      compile 'com.fasterxml.jackson.module:jackson-module-kotlin'
      compile 'com.fasterxml.jackson.datatype:jackson-datatype-joda'
  
      // Joda
      compile 'joda-time:joda-time'
      compile 'org.jadira.usertype:usertype.core:6.0.1.GA'
  
      // Libraries
      compile group: 'org.apache.commons', name: 'commons-text', version: '1.1'
      compile group: 'org.apache.commons', name: 'commons-csv', version: '1.8'
  
      // Kotlin
      compile &quot;org.jetbrains.kotlin:kotlin-stdlib-jdk8:${kotlinVersion}&quot;
      compile &quot;org.jetbrains.kotlin:kotlin-reflect:${kotlinVersion}&quot;
  
      // Mail
      compile group: 'org.springframework.boot', name: 'spring-boot-starter-mail', version: '2.2.4.RELEASE'
      compile group: 'org.apache.commons', name: 'commons-email', version: '1.5'
      compile group: 'javax.mail', name: 'mail', version: '1.5.0-b01'
  
      // AWS
      compile &quot;com.amazonaws:aws-java-sdk-s3:$awsSdkVersion&quot;
      compile group: 'com.amazonaws', name: 'aws-java-sdk-sqs', version: '1.11.558'
      compile group: 'software.amazon.awssdk', name: 's3', version: '2.10.61'
      compile group: 'com.amazonaws', name: 'aws-java-sdk-ec2', version: '1.11.740'
  
      // Support old pw hash lib
      compile group: 'buddy', name: 'buddy-hashers', version: '1.3.0'
  
      // Hazelcast
      compile group: 'com.hazelcast', name: 'hazelcast-spring'
      compile group: 'com.hazelcast', name: 'hazelcast-aws', version: '2.4'
  
      // Swagger
      compile group: 'io.springfox', name: 'springfox-swagger2', version: springfoxVersion
      compile group: 'io.springfox', name: 'springfox-swagger-ui', version: springfoxVersion
  
      // Database
      runtime 'mysql:mysql-connector-java'
      runtime 'org.liquibase:liquibase-core'
      compile 'org.influxdb:influxdb-java:2.13'
  
      // HTML Generation
      compile group: 'org.jetbrains.kotlinx', name: 'kotlinx-html-jvm', version: '0.6.10'
      //  ActiveMQ
      compile group: 'org.messaginghub', name: 'pooled-jms'
  
      // Development
      compile &quot;org.springframework.boot:spring-boot-devtools&quot;
  
      // Testing
      testCompile 'org.springframework.boot:spring-boot-starter-test'
      testCompile 'org.springframework.security:spring-security-test'
      testCompile 'com.h2database:h2'
      testCompile 'io.kotlintest:kotlintest:2.0.7'
      testCompile 'com.nhaarman:mockito-kotlin:1.6.0'
  
  
      // GSON
      compile group: 'com.google.code.gson', name: 'gson', version: '2.8.6'
  }
  
  liquibase {
      activities {
          main {
              changeLogFile '/db/changelog/db.changelog-master.yaml'
              url 'jdbc:mysql://localhost:3307/yellow'
              username 'user'
              password 'secret'
              classpath 'src/main/resources'
          }
      }
  }
  
  
  springBoot {
      buildInfo()
  }
  
  task ebextensions(type: Exec) {
      executable &quot;sh&quot;
      args &quot;-c&quot;, &quot;jar uf build/libs/webcam-service*.jar .ebextensions&quot;
  }
  
  bootJar.finalizedBy ebextensions
  
  afterReleaseBuild.dependsOn bootJar
  
  test {
      minHeapSize = &quot;1024m&quot;
      maxHeapSize = &quot;1024m&quot;
      jvmArgs = [&quot;-Xloggc:build/gclog-%p.log&quot;, &quot;-XX:+PrintGCDetails&quot;]
  }
  </code></pre>
  <p>And the new kotlin one:</p>
  <pre><code>import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
  
  buildscript {
      dependencies {
          classpath(&quot;org.jfrog.buildinfo:build-info-extractor-gradle:4.9.6&quot;)
          classpath(&quot;org.jetbrains.kotlin:kotlin-allopen:1.4.10&quot;)
          // liquibase stuff
          classpath(&quot;mysql:mysql-connector-java:5.1.44&quot;)
          classpath(&quot;org.yaml:snakeyaml:1.19&quot;)
          classpath(&quot;org.liquibase:liquibase-gradle-plugin:1.2.3&quot;) {
              exclude(&quot;org.liquibase&quot;, &quot;liquibase-core&quot;)
          }
          classpath(&quot;org.liquibase:liquibase-core:3.8.9&quot;) // should be in sync with the version included in spring boot
      }
  }
  
  val awsSdkVersion = &quot;1.11.740&quot;
  val springfoxVersion = &quot;2.9.2&quot;
  
  
  plugins {
      id(&quot;org.springframework.boot&quot;) version &quot;2.3.3.RELEASE&quot;
      id(&quot;io.spring.dependency-management&quot;) version &quot;1.0.9.RELEASE&quot;
      id(&quot;net.researchgate.release&quot;) version &quot;2.6.0&quot;
      kotlin(&quot;jvm&quot;) version &quot;1.4.10&quot;
      kotlin(&quot;plugin.spring&quot;) version &quot;1.4.10&quot;
      `maven-publish`
      id(&quot;com.jfrog.artifactory&quot;) version &quot;4.14.1&quot;
      id(&quot;org.liquibase.gradle&quot;) version &quot;2.0.4&quot;
      id(&quot;org.jetbrains.kotlin.plugin.noarg&quot;) version &quot;1.4.10&quot;
      id(&quot;org.jetbrains.kotlin.plugin.jpa&quot;) version &quot;1.4.10&quot;
  }
  
  group = &quot;webcam.yellow.service&quot;
  java.sourceCompatibility = JavaVersion.VERSION_1_8
  
  val mavenUser: String by project
  val mavenPassword: String by project
  val artifactoryRepository = System.getenv(&quot;ARTIFACTORY_REPO&quot;) ?: &quot;snapshots&quot;
  
  
  repositories {
      mavenCentral()
      jcenter()
      maven {
          credentials {
              username = mavenUser
              password = mavenPassword
          }
          url = uri(&quot;https://artifactory.yellow.webcam/artifactory/releases&quot;)
      }
  
      maven {
          credentials {
              username = mavenUser
              password = mavenPassword
          }
          url = uri(&quot;https://artifactory.yellow.webcam/artifactory/snapshots&quot;)
      }
  }
  
  dependencies {
  
      // Spring Boot
      implementation(&quot;org.springframework.boot:spring-boot-starter-web&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-starter-activemq&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-starter-actuator&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-starter-validation&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-starter-security&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-starter-data-jpa&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-starter-mail&quot;)
  
  
      implementation(&quot;io.micrometer:micrometer-registry-influx&quot;)
  
      // Kotlin
      implementation(&quot;org.jetbrains.kotlin:kotlin-stdlib-jdk8&quot;)
      implementation(&quot;org.jetbrains.kotlin:kotlin-reflect&quot;)
      implementation(&quot;org.jetbrains.kotlinx:kotlinx-html-jvm:0.6.8&quot;)
  
      // Yellow Dependencies
      implementation(&quot;webcam.yellow.api:webcam-api:3.2.0&quot;)
      implementation(&quot;webcam.yellow.authentication:messaging-authentication:1.5&quot;)
  
      // AWS
      implementation(&quot;com.amazonaws:aws-java-sdk-s3:$awsSdkVersion&quot;)
      implementation(&quot;software.amazon.awssdk:s3:2.10.61&quot;) // does not yet incorporate all functionality of 1.11.X
      implementation(&quot;com.amazonaws:aws-java-sdk-sqs:$awsSdkVersion&quot;)
      implementation(&quot;com.amazonaws:aws-java-sdk-ec2:$awsSdkVersion&quot;)
  
      // Support old pw hash lib
      implementation(&quot;buddy:buddy-hashers:1.3.0&quot;)
  
      // Jackson
      implementation(&quot;com.fasterxml.jackson.module:jackson-module-kotlin&quot;)
      implementation(&quot;com.fasterxml.jackson.datatype:jackson-datatype-joda&quot;)
  
      // Hazelcast
      implementation(&quot;com.hazelcast:hazelcast-spring&quot;)
      implementation(&quot;com.hazelcast:hazelcast-aws:2.4&quot;)
  
      // Other Libs
      implementation(&quot;org.apache.commons:commons-text:1.1&quot;)
      implementation(&quot;org.apache.commons:commons-csv:1.8&quot;)
  
      implementation(&quot;org.messaginghub:pooled-jms&quot;)
      implementation(&quot;org.influxdb:influxdb-java:2.13&quot;)
      implementation(&quot;org.jetbrains.kotlinx:kotlinx-html-jvm:0.6.10&quot;)
      implementation(&quot;org.springframework.boot:spring-boot-devtools&quot;)
      implementation(&quot;com.google.code.gson:gson:2.8.6&quot;)
      implementation(&quot;joda-time:joda-time&quot;)
      implementation(&quot;org.jadira.usertype:usertype.core:6.0.1.GA&quot;)
  
  
      // Swagger
      implementation(&quot;io.springfox:springfox-swagger2:$springfoxVersion&quot;)
      implementation(&quot;io.springfox:springfox-swagger-ui:$springfoxVersion&quot;)
  
      // Database
      runtimeOnly(&quot;mysql:mysql-connector-java&quot;)
      runtimeOnly(&quot;org.liquibase:liquibase-core&quot;)
      implementation(&quot;org.influxdb:influxdb-java:2.13&quot;)
  
  
      // Test
      testImplementation(&quot;org.springframework.boot:spring-boot-starter-test&quot;)
      testImplementation(&quot;org.springframework.security:spring-security-test&quot;)
      testImplementation(&quot;com.h2database:h2&quot;)
      testImplementation(&quot;com.nhaarman:mockito-kotlin:1.6.0&quot;)
      testImplementation(&quot;io.kotlintest:kotlintest:2.0.7&quot;)
  
  }
  
  tasks.withType&lt;Test&gt; {
      useJUnitPlatform()
  }
  
  springBoot {
      buildInfo()
  }
  
  liquibase {
      activities.register(&quot;main&quot;) {
          arguments = mapOf(
              &quot;changeLogFile&quot; to &quot;/db/changelog/db.changelog-master.yaml&quot;,
              &quot;url&quot; to &quot;jdbc:mysql://localhost:3307/yellow&quot;,
              &quot;username&quot; to &quot;user&quot;,
              &quot;password&quot; to &quot;secret&quot;,
              &quot;classpath&quot; to &quot;src/main/resources&quot;
          )
      }
  }
  
  
  tasks {
      withType&lt;KotlinCompile&gt; {
          kotlinOptions {
              freeCompilerArgs = listOf(&quot;-Xjsr305=strict&quot;)
              jvmTarget = &quot;1.8&quot;
          }
      }
  
      bootJar {
          from(&quot;./.ebextensions&quot;) { into(&quot;.ebextensions&quot;) }
          launchScript()
      }
  
      afterReleaseBuild {
          dependsOn(bootJar)
      }
  
      test {
          minHeapSize = &quot;1024m&quot;
          maxHeapSize = &quot;1024m&quot;
          jvmArgs = listOf(&quot;-Xloggc:build/gclog-%p.log&quot;, &quot;-XX:+PrintGCDetails&quot;)
      }
  }
  </code></pre>
  <p>Some things needed different handling, like moving some plugins from the classpath to the plugins section, the .ebsextensions are handled differently, and Jacoco isn't in there anymore because we  never really used those reports in jenkins. All in all, pretty normal. Everything works, except for this one thing in the unit tests.</p>
  <p>Here's a class initialisation and one of the tested methods:</p>
  <pre><code>@Service
  class ImageService(private val imageRepository: ImageRepository,
                     private val imageSetRepository: ImageSetRepository,
                     private val permissionService: PermissionService,
                     private val s3Service: S3Service,
                     private val entityManager: EntityManager,
                     private val panoFeedRepository: PanoFeedRepository,
                     private val panoImageRepository: PanoImageRepository,
                     private val jobFacade: JobFacade,
                     private val jdbcTemplate: NamedParameterJdbcTemplate,
                     transactionManager: PlatformTransactionManager,
                     @Value(&quot;\${service.cleanup.enabled}&quot;)
                     private val cleanupEnabled: Boolean) {
  
  
      private val readTransaction = org.springframework.transaction.support.TransactionTemplate(
              transactionManager,
              DefaultTransactionDefinition().apply {
                  isReadOnly = true
                  propagationBehavior = TransactionDefinition.PROPAGATION_REQUIRES_NEW
              })
      private val writeTransaction = org.springframework.transaction.support.TransactionTemplate(
              transactionManager,
              DefaultTransactionDefinition().apply {
                  isReadOnly = false
                  propagationBehavior = TransactionDefinition.PROPAGATION_REQUIRES_NEW
              })
  
  
      /**
       * Deletes all passed images from the database while removing them from S3 asynchronously.
       * Does not check any permissions.
       * This opens its own write transaction, which could take a long time. If individual transactions take too long,
       * invoke this method multiple times with smaller chunks to avoid issues with table locking.
       * Since this is potentially long-running, commit preceding transactions beforehand.
       */
      @Transactional(propagation = Propagation.NEVER)
      internal fun delete(images: List&lt;ImageIdAndImageKeyAndImagePreviewKey&gt;) {
          images.chunked(500).forEach { chunk -&gt;
              log.debug(&quot;Deleting s3 files of {} images&quot;, chunk.size)
              jobFacade.executeAsync {
                  val keysToDelete = chunk
                          .flatMap { listOf(it.getImageKey(), it.getImagePreviewKey()) }
                          .asSequence()
                          .filterNotNull()
                          .filter { !it.isBlank() }
                          .toList()
  
  
                  s3Service.deleteImages(keysToDelete)
                  log.trace(&quot;deleted {} s3 files of {} images&quot;, keysToDelete.size, chunk.size)
              }
              log.debug(&quot;Deleting {} images from database&quot;, chunk.size)
              writeTransaction.execute {
                  imageRepository.deleteAllByIdIn(chunk.map { it.getId()!! })
              }   // &lt;---- Exception happens in here
              log.trace(&quot;Deleted {} images from database&quot;, chunk.size)
          }
      }
  }
  </code></pre>
  <p>And here's a test setup and a test that tests tat method:</p>
  <pre><code>class ImageServiceTest {
  
      private val imageRepository: ImageRepository = mock()
      private val imageSetRepository: ImageSetRepository = mock()
      private val permissionService: PermissionService = mock()
      private val s3Service: S3Service = mock()
      private val hazelCastInstance: HazelcastInstance = mock()
      private val panoFeedRepository: PanoFeedRepository = mock()
      private val panoImageRepository: PanoImageRepository = mock()
      private val jdbcTemplate: NamedParameterJdbcTemplate = mock()
      private val transactionManager: PlatformTransactionManager = mock()
      private val jobFacade: JobFacade
  
      private val sut: ImageService
  
      init {
          val imap: IMap&lt;String, String&gt; = mock()
          `when`(hazelCastInstance.getMap&lt;String, String&gt;(any())).thenReturn(imap)
          jobFacade = JobFacade(hazelCastInstance)
          sut = ImageService(
                  imageRepository, imageSetRepository, permissionService, s3Service, mock(),
                  panoFeedRepository, panoImageRepository, jobFacade, jdbcTemplate, transactionManager, false)
      }
  
      @Test
      fun `deleting multiple images invokes S3Service in chunks and deletes images from DB`() {
          val user = TestModelFactory.userWithOneCameraAndTablePermission()
          val table = user.tablePermissions.first().table!!
          val images = (1..800).map { TestModelFactory.image(table = table) }
  
          sut.delete(images.map { ImageIdentifiers(it.id, it.imageKey, it.imagePreviewKey) })
          Thread.sleep(2000) //leave enough time for S3 uploads to finish
  
          val idChunks = images.map { it.id }.chunked(500)
          verify(imageRepository, times(1)).deleteAllByIdIn(eq(idChunks[0]))
          verify(imageRepository, times(1)).deleteAllByIdIn(eq(idChunks[1]))
          verify(s3Service, times(2)).deleteImages(any())
      }
  </code></pre>
  <p>And finally, the error:</p>
  <pre><code>java.lang.NullPointerException: Parameter specified as non-null is null: method webcam.yellow.service.service.ImageService$delete$$inlined$forEach$lambda$2.doInTransaction, parameter it
  </code></pre>
  <p>Now, there's several things I'm currently looking into about what might go wrong, involving the mocked TransactionManager and the non-mocked TransactionTemplates... But the <em>really frustrating</em> thing is that this simply didn't happen before migrating the build script, and nothing else changed.</p>