anchor.diff:  ---
   layout: post
   title:  Switching to Java 11/17
  -date:   2022-04-22 12:00:00 +0100
  +date:   2022-05-04 17:15:00 +0100
   tags: [ community, news ]
   author: vjuranek
   ---
   
   As you probably noticed, we have started work on Debezium 2.0.
   One of https://issues.redhat.com/browse/DBZ-3899[the planned changes] for the 2.0 release is https://issues.redhat.com/browse/DBZ-4949[to switch to Java 11 as a baseline].
  -While some Java implementations still supports Java 8, some other Java 8 distributions already reached their end of life/support.
  -Users are moving to Java 11 anyway as surveys like https://newrelic.com/resources/report/2022-state-of-java-ecosystem[New Relic State of the Java Ecosystem Report] indicates.
  -It not only matter of support.
  -Java 11 comes with various performance improvements, useful tools like Java Flight Recorder and Java Mission Control, which were open sourced in Java 11, and more.
  -So it's time to start thinking about using a more recent JDK, and the new major Debezium release is a natural milestone when to do the switch.
  +While some Java build providers still support Java 8, other Java 8 distributions already reached their end of life/support.
  +Users are moving to Java 11 anyways, as surveys like New Relic's https://newrelic.com/resources/report/2022-state-of-java-ecosystem[State of the Java Ecosystem Report] indicate.
  +But it is not only matter of support:
  +Java 11 comes with various performance improvements, useful tools like JDK Flight Recorder, which was open-sourced in Java 11, and more.
  +So we felt it was about time to start thinking about using a more recent JDK as the baseline for Debezium, and the new major release is a natural milestone when to do the switch.
   
   +++<!-- more -->+++
   
  -Since early releases of Debezium 2.0, from the very first one `Alpha1`, Debezium bits will be compiled to Java 11 byte code.
  +Starting with the first release of Debezium 2.0, link:/blog/2022/04/28/debezium-2.0-alpha1-released/[2.0.0.Alpha1], Debezium bits will be compiled to Java 11 byte code.
   Therefore, Java 11 will be required to run Debezium in the next major update.
  -Also, if you use any of the Debezium bits in your project, you will have to switch to Java 11.
  +Also, if you use any of the Debezium bits as a library in your project
  +(using the Debezium link:/documentation/reference/stable/development/engine.html[embedded engine]), you will have to switch to Java 11.
   
   But wait, what does Java 11/17 in the title mean?
  -Is it there just to scare you or we are going to actually switch to Java 17?
  +Is it there just to scare you, or we are going to actually switch to Java 17 right away?
   
   <dramatic pause here>
   
   No, we don't want to scare you.
   We are actually planning to switch to Java 17, but only for the test suite.
   Please note that both Java 11 and 17 are long term support (LTS) releases.
  -We don't want to Java 17 directly, as it can be an issue for substantial amount of Debezium users, 
  -e.g. aforementioned New Relic report shows that most of the users are still on Java 11 and we don't want to exclude them.
  -However, using Java 17 for test doesn't affect users in any way and will allow us to use more recent Java features in the tests, like e.g.
  -* https://openjdk.java.net/jeps/378[text block] which can make more easy writing of JSON strings,
  -* https://openjdk.java.net/jeps/384[records] which can improve readability of the stream operations heavily used in our tests,
  -* https://openjdk.java.net/jeps/361[switch expressions] and more.
  +We don't want to move Java 17 for the actual Debezium artifacts just yet, as it can be an issue for substantial amount of Debezium users;
  +as e.g. the aforementioned New Relic report shows that most of the users are still on Java 11 and of course we don't want to exclude them.
  +However, using Java 17 for tests doesn't affect users in any way, and will allow us to use some more recent Java features in the tests, like e.g.
  +* https://openjdk.java.net/jeps/378[text blocks], which for instance simplify the usage of multi-line JSON or SQL strings,
  +* https://openjdk.java.net/jeps/384[records], which can improve readability of the stream operations heavily used in our tests,
  +* https://openjdk.java.net/jeps/361[switch expressions], and more.
  +
  +Pretty sweet, right?
   
   == Implementation
   
  -Setting different byte code level for code and tests is pretty easy with Maven, 
  -you just need to set following system properties:
  +Setting different byte code levels for code and tests is pretty easy with Maven,
  +you just need to set the following properties:
   
   [source, xml]
   ----
  
positive.code: <maven.compiler.testRelease>17</maven.compiler.testRelease>
  ----
  
  Please note using `release` option instead of legacy `source` and `target`.
  When target release is different from the source, using `source` and `target` can cause you unexpected issues.
  See e.g. Gunnar's blog post https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/[ByteBuffer and the Dreaded NoSuchMethodError] for more details.
  
  After switching to Java 11, the [Maven Checkstyle plug-in](https://maven.apache.org/plugins/maven-checkstyle-plugin/) and the [ImpSort plug-in](https://code.revelc.net/impsort-maven-plugin/) (a plug-in which takes care of proper import ordering) started to fail.
  However, bumping their versions to the latest releases has solved all the issues.
  
  This was the easy part.
  The most difficult part was the Debezium [connector for Apache Cassandra](https://debezium.io/documentation/reference/stable/connectors/cassandra.html).
  
  === Cassandra connector
  
  The Cassandra connector now provides support for Cassandra 3 as well as for Cassandra 4.
  Cassandra 4 https://cassandra.apache.org/doc/4.0/cassandra/new/java11.html[works with Java 11], 
  but running Cassandra 3 with Java 11 is not possible (or at least requires some hacking).
  The existing test implementation for this connector didn't run Cassandra in a container as we do it in tests for all other DB connectors, but instead runs Cassandra in embedded mode, i.e. within the same JVM and process as the tests themselves.
  Therefore if you wanted to run the tests with Java 11 (or 17), tests for the Cassandra 3 connector module would fail.
  The obvious solution is to run Cassandra in a container with Java 8.
  This sounds good, but this approach has one pitfall.
  The Cassandra connector needs access to Cassandra log files as it reads the dedicated log file to obtain CDC events, so the tests need to access Cassandra files in the container.
  This can be solved quite easily using a temporary directory, for instance within the `target` directory, and mount it as a volume into the container running Cassandra.
  Cassandra running in the container can later on use this mounted volume for storing its data.
  The real issue starts when you try to do the cleanup after the tests.
  As Cassandra runs  in the container under a dedicated user named`cassandra`, which is very likely not present on the test machine (or with a different UID/GID), 
  cleanup fails when it tries to delete the temporary directory with Cassandra files.
  These files were created in that temporal directory mounted into the container and not in Docker FS overlay, so that are present in `target` directory.
  As the files were created by `cassandra` user, which is very likely different user than one who runs the tests, user running the tests has insufficient rights to delete files created by `cassandra` user.
  Trying to delete them from Cassandra's container on Cassandra exit in some wrapper script turned out to be quite cumbersome and not very reliable.
  The most promising solution proved to involve starting a second container with the same `cassandra` user with access to the mounted volume and cleaning up the files after the first Cassandra container had already stopped.
  
  We considered two option for running containers:
  
  * https://dmp.fabric8.io/[Fabric8 Docker Maven plugin]
  * https://www.testcontainers.org/[Testcontainers]
  
negative.code: We use the Fabric8 plugin in the rest of the project, which suggests to use it also in this case to have uniformity across the project.
  On the other hand, using Testcontainers would make tests more convenient for the developers (who actually use tests after all!),
  as it allows to run the tests directly from IDE without starting the container manually.
  
  In the end, the decision was driven by the fact that running a cleanup container is not possible with the Fabric8 plugin.
  Maven doesn't allow to execute different configurations in the same phase 
  and therefore it's not possible to stop the Cassandra container in the `post-integration-test` phase and at the same time run a cleanup container in this phase.
  