Chapter 3. Configuring Java applications
Chapter 3. Configuring Java applications To enable Cryostat to gather, store, and analyze Java Flight Recorder (JFR) data about target applications that run on Java Virtual Machines (JVMs), you must configure the applications so that Cryostat can detect and connect to them. You can configure the applications in any of the following ways: Cryostat agent You can use the Cryostat agent component for detection and connectivity, which is implemented as a Java Instrumentation Agent and acts as a plug-in for applications that run on the JVM. The Cryostat agent provides an HTTP API that the Cryostat server can use as an alternative to an application's JMX port. By attaching a properly configured Cryostat agent to the workload applications that you deploy, you can use the full Cryostat feature set without any need for the target applications to expose a JMX port. Note Before Red Hat build of Cryostat 2.4, the Cryostat agent provided a read-only HTTP API that supported a limited set of JFR operations only. The Cryostat agent's HTTP API can offer the following benefits compared to a JMX port: Greater security due to the reduced API surface area Deployment flexibility due to the Cryostat agent's dual role as a Cryostat discovery plug-in For more information, see Working with the Cryostat agent . Remote Java Management Extensions (JMX) connections You can configure your target applications to allow Java Management Extensions (JMX) connections. This type of configuration uses an OpenShift Service for detection and JMX for connectivity. JMX is a standard feature on a JVM with which you can monitor and manage target applications that run on the JVM. For Cryostat to use JMX, you must enable and configure JMX when you start the JVM, because Cryostat requires the target applications to expose a JMX port. Cryostat communicates with the target applications over this JMX port to start and stop JFR recordings and to pull JFR data over the network, enabling Cryostat to store and analyze this JFR data. Remote monitoring requires security to ensure that unauthorized persons cannot access application. Cryostat prompts you to enter your credentials before Cryostat can access any of the application's JFR recordings. For more information, see Configuring applications by using JMX connections . Cryostat agent and JMX hybrid You can configure your target applications to use a hybrid approach where you use both the Cryostat agent and JMX. With this approach, you use the Cryostat agent to detect the target applications and JMX to expose the JFR data to Cryostat, which allows for more flexibility. For example, you can use the agent to detect the applications without needing to depend on specific port numbers and also use the JMX connections to start and stop JFR flight recordings on demand. If the Cryostat agent detects that JMX is also configured on your application, the agent publishes itself to the Cryostat server with both agent HTTP API definitions and JMX URL definitions. In this situation, you can use whichever configuration option you prefer. For more information, see Configuring applications by using the Cryostat agent and JMX connections . 3.1. Working with the Cryostat agent The Cryostat agent is implemented as a Java Instrumentation Agent that acts as a plug-in for applications that run on the JVM. The Cryostat agent provides an HTTP API that offers greater deployment flexibility than a JMX port due to the agent's dual role as a discovery plug-in. You can configure your target applications to use the agent's HTTP API for both detection and connectivity with Cryostat. 3.1.1. Cryostat agent features Red Hat build of Cryostat offers different distributions of the Cryostat agent and supports different ways of deploying and using the agent. Consider the following information to help you determine the type of Cryostat agent deployment and use cases that best suit your needs. 3.1.1.1. Agent JAR file distributions Red Hat build of Cryostat 3.0 distributes two different variations of the Cryostat agent's JAR file: An all-in-one "shaded" JAR file that is self-contained and includes the agent code and all of its dependencies This "shaded" JAR file provides the most convenient form of Cryostat agent to include in your existing applications, because you need to include only one additional agent JAR file. This is a common distribution pattern for similar agents and tools. Before Cryostat 2.4, the "shaded" JAR file was the only available distribution of the Cryostat agent. The "shaded" JAR file name is in the form cryostat-agent-0.4. X .redhat- xxxxx -shaded.jar , where 0.4. X and xxxxx represent the agent version and build number, respectively. A standard JAR file that contains the agent code without any dependencies This type of JAR file is useful if you know that dependency conflicts exist between the agent and your workload applications. If you intend to apply your own strategies to provide the correct versions of each dependency to satisfy both the agent and your applications' requirements, you can use the standalone JAR file. The standard JAR file name is in the form cryostat-agent-0.4. X .redhat- xxxxx .jar , where where 0.4.X and xxxxx represent the agent version and build number, respectively. Important If you want to use the Cryostat agent, the all-in-one "shaded" JAR file is the preferred way to deploy the agent. Do not use the standard JAR file unless you intend to apply your own strategies to resolve any dependency conflicts between the agent and your workload applications. The agent JAR files are available for download from the Red Hat Maven Repository . 3.1.1.2. Static or dynamic attachment to the JVM Depending on your requirements, the Cryostat agent can use a static or dynamic approach for attaching to the JVM. Static attachment to the JVM You can enable your workload application's JVM to load and initialize the Cryostat agent at JVM startup. This static attachment approach requires that you configure the application to pass the -javaagent JVM flag with the path to the Cryostat agent's JAR file (for example, ‐javaagent:/deployment/app/lib/cryostat-agent-shaded.jar ). Once the Cryostat agent's basic initialization is complete, your workload application's normal startup process begins as usual. Depending on your workload application, the static attachment approach might require that you set one or more environment variables or add an argument to the argLine parameter. However, in some cases, you might need to reconfigure, rebuild, and redeploy your application. Static attachment also requires that you restart the application JVM, which can cause application downtime. Note Before Cryostat 3.0, static attachment through the -javaagent JVM flag was the only way to enable the Cryostat agent to attach and run on a target application. For more information, see Configuring applications by using the Cryostat agent . Dynamic attachment to the JVM From Cryostat 3.0 onward, the Cryostat agent can attach dynamically to an application JVM that is already running without requiring an application restart. This dynamic attachment feature has the following requirements: You must ensure that the agent's JAR file is copied to the JVM's file system (for example, by using the oc cp command). You must be able to run the agent as a separate process on the same host or within the same application (for example, by using the oc exec command). The dynamic attachment feature supports ad hoc one-time profiling or troubleshooting workflows where you might not need the agent to be attached every time the JVM starts. Dynamic attachment also suits situations where you cannot or do not want to reconfigure your application for the sole purpose of attaching the agent. Because the agent can attach to a running JVM without requiring an application restart, this also means there is no application downtime. For more information, see Launching the Cryostat agent as a standalone process for dynamic attachment to the JVM . 3.1.1.3. Options for including the agent's JAR file into your workload application Depending on your requirements, you can include the Cryostat agent's JAR file into the workload application in different ways: For dynamic one-time attachment to a running JVM, you can copy the agent's JAR file to the JVM's file system by using the oc cp command. For static attachment to a JVM at startup, you can use any of the following options: The simplest option is to add the JAR file to your application's dependencies in the pom.xml or build.gradle file. Your build tool (Maven or Gradle) downloads the JAR file for inclusion in your application build output. You can use a Maven plug-in, such as the maven-dependency-plugin , to provide more fine-grained control over the download and inclusion of the JAR file in the application build output. You can create a PersistentVolume storage volume that contains the JAR file. Then reconfigure your application's Deployment/DeploymentConfig to mount the PersistentVolume and use -javaagent:/path/to/persistentvolume/cryostat-agent-shaded.jar . The exact way to accomplish this task depends on the type of PersistentVolume provider you enabled in your OpenShift cluster. Once the Cryostat agent is successfully added to your application container and loaded, your application's stdout and console logs start displaying log messages from the Cryostat agent. 3.1.1.4. Agent configuration properties You can specify configuration properties for the Cryostat agent in either of two ways: Use JVM system property flags on the application (for example, -Dcryostat.agent.api.writes-enabled=true ). Use environment variables by making all letters upper case and replacing any punctuation with underscores (for example, CRYOSTAT_AGENT_API_WRITES_ENABLED=true ). Required agent properties You must configure the following properties to enable the Cryostat agent to operate successfully: cryostat.agent.baseuri This specifies the URL location of the Cryostat server back end that the Cryostat agent advertises itself to (that is, the internal OpenShift Service object) such as https://my-cryostat.my-namespace.svc.cluster.local . cryostat.agent.callback This specifies the URL location of the Cryostat agent instance or application itself. Cryostat uses this URL to perform health checks and request data from the agent. You can use the OpenShift/Kubernetes Downward API to determine this dynamically by setting this property to a value such as https://USD(POD_IP):9977 . For more information, see the Kubernetes Downward API documentation on status.podIP . cryostat.agent.authorization This specifies a header value that the Cryostat agent can include in API requests to the Cryostat server (for example, Bearer abcd1234 ). This is set to an empty string by default. Note This property takes precedence over cryostat.agent.authorization.type and cryostat.agent.authorization.value . If you specify a value for this property, the agent uses this value in the header of API requests. Otherwise, the agent uses the values that you specify for cryostat.agent.authorization.type and cryostat.agent.authorization.value . cryostat.agent.authorization.type This specifies an authorization type that is used in conjunction with the cryostat.agent.authorization.value property. The authorization type performs a mapping of the cryostat.agent.authorization.value to produce a header that the Cryostat agent can include in API requests to the Cryostat server. Note The agent ignores this property if you specify a value for the cryostat.agent.authorization property. Valid authorization types are as follows: basic This encodes the authorization value by using Base64 to produce a Basic base64(value) header. bearer This directly embeds the authorization value into a Bearer value header. kubernetes This reads the authorization value as a file location to produce a Bearer fileAsString(value) header. none This produces no header. auto This first attempts the same behavior as the kubernetes option. Otherwise, if unsuccessful, this reverts to none to produce no header. This property is set to auto by default. cryostat.agent.authorization.value This specifies an authorization value that is used in conjunction with the cryostat.agent.authorization.type property. The authorization value is mapped by the authorization type to produce a header that the Cryostat agent can include in API requests to the Cryostat server. Note The agent ignores this property if you specify a value for the cryostat.agent.authorization property. Consider the following guidelines: If the authorization type is basic , specify an authorization value that matches the unencoded basic credentials (for example, user:pass ). If the authorization type is bearer , specify an authorization value that matches the token that needs to be presented. If the authorization type is kubernetes , specify an authorization value that matches the file-system path to the service account token secret file If the authorization type is none , any value that you specify is ignored. By default, this property is set to /var/run/secrets/kubernetes.io/serviceaccount/token , which is the standard location for Kubernetes service account token secret files. Optional agent properties Depending on your setup requirements, you can also configure the following agent properties: cryostat.agent.api.writes-enabled This indicates whether the Cryostat agent allows write operations. This is set to false by default. If you want the Cryostat agent to accept requests to start, stop, or delete JFR flight recordings, you must set this property to true . Note Even if this property is set to false , the agent can still fulfill requests to list flight recordings or download individual recording files. cryostat.agent.webserver.port This specifies the HTTP port number that the agent uses to bind its HTTP API (by default, 9977). If this conflicts with an existing port that your application or another tooling agent uses, you must specify a different port number. cryostat.agent.app.name This specifies a label for identifying which application this Cryostat agent instance is attached to (by default, cryostat-agent ). You could use the Downward API metadata.name or metadata.labels['app'] fields for this. For more information, see the Kubernetes Downward API documentation Kubernetes Downward API documentation. The preceding list describes only a subset of the available configuration properties for the Cryostat agent. For a full list of agent properties, see Agent Properties . 3.1.2. Configuring applications by using the Cryostat agent You can use the Cryostat agent, implemented as a Java Instrumentation Agent, to configure target applications, so that Cryostat can detect the applications, collect data, and send the data to Cryostat for analysis. You can also optionally enable the Cryostat agent to accept requests from the Cryostat server to start, stop, and delete JFR recordings. Note This procedure is relevant if you want to use the static attachment feature, which allows your workload application's JVM to load and initialize the Cryostat agent at JVM startup. If you want the Cryostat agent to attach to a running JVM on a one-time basis, see Launching the Cryostat agent as a standalone process for dynamic attachment to the JVM instead. Important As described in Agent JAR file distributions , Red Hat provides two different variations of the Cryostat agent either as an all-in-one "shaded" JAR file or as a standard JAR file. The all-in-one "shaded" JAR file is the preferred way to deploy the Cryostat agent. Do not use the standard JAR file unless you intend to apply your own strategies to resolve any dependency conflicts between the agent and your workload applications. The following procedure describes how to install the "shaded" JAR file distribution of the Cryostat 3.0 agent. The latest "shaded" JAR file is available for download from the Red Hat Maven repository . The "shaded" JAR file name is in the form cryostat-agent-0.4. X .redhat- xxxxx -shaded.jar , where 0.4. X and xxxxx represent the agent version and build number, respectively. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository . As described in Options for including the agent's JAR file into your workload application , the Cryostat 3.0 agent supports different options for including the agent's JAR file into your workload applications. The following procedure describes how to add the "shaded" JAR file to your application's dependencies in the pom.xml or build.gradle file. Prerequisites Logged in to your Cryostat web console. Installed JDK version 11 or later. Procedure Install the Cryostat agent. Choose one of the following options, depending on your application build: Using Maven: Update the application pom.xml file with the Cryostat agent JAR file information. Example pom.xml <project> ... <repositories> <repository> <id>redhat-maven-repository</id> <url> https://maven.repository.redhat.com/earlyaccess/all/</url> </repository> </repositories> ... <build> <plugins> <plugin> <artifactId>maven-dependency-plugin</artifactId> <version>3.3.0</version> <executions> <execution> <phase>prepare-package</phase> <goals> <goal>copy</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>io.cryostat</groupId> <artifactId>cryostat-agent</artifactId> <version>0.4. X .redhat- xxxxx </version> <classifier>shaded</classifier> </artifactItem> </artifactItems> <stripVersion>true</stripVersion> </configuration> </execution> </executions> </plugin> </plugins> ... </build> ... </project> Note In the preceding example, replace 0.4. X .redhat- xxxxx with the latest version and build number of the Cryostat agent. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository . The time you build your application, the Cryostat agent JAR file is available at target/dependency/cryostat-agent-shaded.jar . Using Gradle: Update the build.gradle file. Example build.gradle file repositories { ... maven { url "https://maven.repository.redhat.com/earlyaccess/all/" credentials { username "myusername" password "mytoken" } } } How you package the agent JAR file into the application depends on the Gradle plug-ins that you use for the build. For example, if you are using the Jib plug-in, update the build.gradle file as follows: Example build.gradle file plugins { id 'java' id 'application' id 'com.google.cloud.tools.jib' version '3.3.1' id 'com.ryandens.javaagent-jib' version '0.5.0' } ... dependencies { ... javaagent 'io.cryostat:cryostat-agent:0.4. X .redhat- xxxxx :shaded' Note In the preceding example, replace 0.4. X .redhat- xxxxx with the latest version and build number of the Cryostat agent. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository . Update the Docker file. The following example uses the JAVA_OPTS environment variable to pass the relevant JVM information. Example ... COPY target/dependency/cryostat-agent-shaded.jar /deployments/app/ ... ENV JAVA_OPTS="-javaagent:/deployments/app/cryostat-agent-shaded.jar" Rebuild the container image that is specific to your application. To supply the JVM system properties or environment variables that you need to configure the Cryostat agent, push the updated image and then modify your application deployment. Example apiVersion: apps/v1 kind: Deployment ... spec: ... template: ... spec: containers: - name: sample-app image: docker.io/myorg/myapp:latest env: - name: CRYOSTAT_AGENT_APP_NAME value: "myapp" # Replace this with the Kubernetes DNS record # for the Cryostat Service - name: CRYOSTAT_AGENT_BASEURI value: "http://cryostat.mynamespace.mycluster.svc:4180" - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CRYOSTAT_AGENT_CALLBACK value: "http://USD(POD_IP):9977" 1 # Replace "abcd1234" with a plain-text authentication token - name: CRYOSTAT_AGENT_AUTHORIZATION 2 value: "Bearer abcd1234" - name: CRYOSTAT_AGENT_API_WRITES_ENABLED 3 value: true ports: - containerPort: 9977 protocol: TCP resources: {} restartPolicy: Always status: {} <1>: Port number 9977 is the default HTTP port that the agent exposes for the internal web server that services Cryostat requests. You can change this port number if it conflicts with your target application into which the agent is installed. <2>: The CRYOSTAT_AGENT_AUTHORIZATION value shows the credentials that the agent includes in the API requests to Cryostat to advertise its own presence or to push JFR data. You can also create a Kubernetes Service Account for this purpose and replace abcd1234 with the plain-text authentication token associated with the service account. <3>: The CRYOSTAT_AGENT_API_WRITES_ENABLED variable is set to false by default. If you want the Cryostat agent to accept requests from the Cryostat server to start, stop, or delete JFR flight recordings, you must set this variable to true . 3.1.2.1. Configuring the Cryostat agent to trust the Cryostat server When you use the Cryostat agent with a Cryostat instance that has cert-manager integration enabled, the Cryostat agent communicates with Cryostat over a secure HTTPS connection. In this situation, the Cryostat Operator uses cert-manager to generate a self-signed certificate authority (CA) certificate, which is stored within a secret. You must configure the Cryostat agent to trust this CA certificate. Procedure Create a Cryostat CR that defines the namespaces for your Cryostat instance and your target applications. For example, if you want to create a Cryostat CR in the cryostat namespace that is configured to connect to target applications in the apps namespace, enter the following details: After Cryostat becomes available, to obtain the CA certificate secret in your target application's namespace, enter the following commands: In the preceding example, replace apps with the namespace of your target application, and replace cryostat/cryostat-sample with the namespace and name of your Cryostat instance. Also, ensure that the namespace and name of your Cryostat instance are separated by a forward slash ( / ). The preceding command produces output similar to the following: As shown in the preceding example, the secret name contains a hash suffix to prevent conflicts with other Cryostat instances on your cluster. Configure the Cryostat agent to trust Cryostat's CA certificate by creating an init container to import the certificate into a truststore. Note This step assumes that you have a target application named my-app in the apps namespace that has the Cryostat agent installed and is otherwise properly configured. In the following examples, replace any occurrences of my-app and apps with the name and namespace of your target application, as appropriate. Create a volume for the Cryostat CA secret in your deployment by using the secret name that you obtained in Step 2 . For example: The preceding command produces output similar to the following: Create an emptyDir volume to share the truststore between the init container and your application container: For example: The preceding command produces output similar to the following: Add an init container to the deployment YAML file for your target application. For example: In the preceding example, replace <my password> with the password that you want to use. Define the javax.ssl.trustStore Java system property in your application. For example: In the preceding example, replace <my password> with the password that you specified in the step. Note How you set Java system properties depends on how your application is built and which base image you used. The preceding example assumes that you are using an application that was built with the OpenJDK UBI images. 3.1.3. Launching the Cryostat agent as a standalone process for dynamic attachment to the JVM If you want the Cryostat agent to attach dynamically to an application JVM that is already running, you can launch the agent as a standalone Java process. Note This procedure is only relevant if you want to use the dynamic attachment feature, which allows the Cryostat agent to attach to a running JVM on an ad hoc one-time basis. If you want your workload application's JVM to load and initialize the Cryostat agent at JVM startup, see Configuring applications by using the Cryostat agent . Prerequisites Copied the agent's JAR file to the JVM's file system by using the oc cp command. Note The latest "shaded" JAR file is available for download from the Red Hat Maven repository . The "shaded" JAR file name is in the form cryostat-agent-0.4. X .redhat- xxxxx -shaded.jar , where 0.4. X and xxxxx represent the agent version and build number, respectively. Procedure Enter the following command: USD java -jar target/ <agent_jar_file> <pid> In the preceding command, replace <agent_jar_file> with the agent's JAR file name and replace <pid> with the process ID (PID) of the JVM that you want the agent to attach to. For example: USD java -jar target/cryostat-agent-0.4.1.redhat-00001-shaded.jar 1234 Note The preceding example is shown for illustration only and might not reflect the latest available version and build of the "shaded" JAR file. When you run the preceding command, the agent process uses its Attach providers to look up the specified PID. If the specified PID is found, the agent process attaches to this PID and attempts to load the agent's JAR file into this JVM, which then bootstraps into the normal agent launch process. Agent launch behavior based on the PID value Consider the following guidelines for agent launch behavior depending on the PID value: If you specify an invalid PID, the agent cannot launch successfully. If you specify a wildcard asterisk ( * ) as the PID value, the agent's JAR file attempts to bootstrap into every JVM that it finds. If you specify 0 as the PID value or if you do not specify any PID value, the agent checks if exactly one candidate JVM is available. If only one JVM is available, the agent attempts to bootstrap into this JVM. If multiple JVMs or no JVMs are available, the agent cannot launch successfully. Late-binding configuration options When launching the Cryostat agent as a standalone process, you can also specify additional late-binding configuration options to the agent launcher by using command-line options. For example: For information about the available options and their behavior, run the java -jar target/ <filename> .jar -h help command, where <filename> represents the name of the "shaded" agent JAR file that you copied. As shown in the preceding example, when launching the Cryostat agent as a standalone process, ensure that you specify the -D flags for any late-binding configuration options after the -jar <path_to_jarfile> argument. The position of the -D flags is important here: If you specify the -D flag for a system property before the -jar <path_to_jarfile> argument, this property is set on the standalone launcher JVM process. This could lead to unexpected issues when launching the agent as a standalone process. If you specify the -D flags for system properties after the -jar <path_to_jarfile> argument, these properties are passed as arguments to the agent launcher. When the launcher bootstraps the agent into the host JVM, the agent applies these properties onto the JVM at agent startup. As shown in the preceding example, ensure that you specify any required agent properties such as cryostat.agent.baseuri , cryostat.agent.callback , and cryostat.agent.authorization as configuration options to the agent launcher. For more information about configuration properties, see Agent configuration properties . 3.2. Configuring applications by using JMX connections For Cryostat to detect and communicate with your target Java applications, you can configure the applications to allow remote Java Management Extensions (JMX) connections. Prerequisites Logged in to your Cryostat web console. Created a Cryostat instance in your project. Procedure To enable remote JMX connections, complete the following steps: On your application, define the following Java system property: Note To add the -Dcom.sun.management.jmxremote.port=<port_num> property without having to rebuild the target application, you can set the JAVA_OPTS_APPEND environment variable on the application. JAVA_OPTS_APPEND is an environment variable that is used by Red Hat Universal Base Images (UBI) only. If you use Red Hat UBI to build application images, set the JAVA_OPTS_APPEND variable at build time in the application Docker file, or at runtime by running the following command: If you do not use Red Hat UBI to build application images, refer to the documentation for your base image for information about how to add the Java system properties at build time or runtime. Specify that the application listens for remote JMX connections by allowing traffic to the application. Use an Red Hat OpenShift Service and specify the following values for its remote JMX port: Example service.yaml apiVersion: v1 kind: Service ... spec: ports: - name: "jfr-jmx" port: 9091 targetPort: 9091 ... Secure the remote JMX connections: Enable and configure authentication and SSL/TLS for the remote JMX connections in your application: Configure Cryostat to trust the application TLS certificate. Create a secret for the application in the same namespace as your Cryostat application and configure Cryostat to refer to the secret. To create a secret for the certificate, run the following command: Note The certificate must be in a .pem file format. When you create your Cryostat instance, add the secret to the list of trusted TLS certificates. For more information, see Configuring TLS certificates . To allow your applications to verify that Cryostat is connecting to them by a means other than password authentication, enable TLS client authentication: Note TLS client authentication requires the cert-manager operator for Red Hat OpenShift. If you use TLS client authentication for remote JMX connections, the application truststore must contain a Cryostat certificate. The Cryostat operator cert-manager integration creates a self-signed certificate for the Cryostat deployment. This certificate is located in the <cryostat>-tls secret, where <cryostat> is the name of the Cryostat instance you created. Note The cert-manager Operator also places a Java keystore truststore in the secret. To mount this truststore in your application deployment, run the following command, replacing "<myapp>" with the name of your application deployment and "<cryostat>" with the name of your Cryostat instance: The Cryostat operator generates the truststore password, which you can find in the <cryostat>-keystore secret. To mount this as an environment variable in your application deployment, run the following command: Configure the Java arguments for the container. Run the following commands: Warning If you deployed Cryostat and your applications in a testing environment, you might want to configure the target applications without any JMX or TLS authentication. You can do so by using the following set of Java system properties, however, this configuration is not secure and not recommended. Additional resources Configuring TLS certificates Storing and managing JMX credentials 3.3. Configuring applications by using the Cryostat agent and JMX connections You can configure target applications that run on Java Virtual Machines (JVM) to use a combination of the Cryostat agent and Java Management Extensions (JMX) connections to detect and communicate with the target applications. You use the Cryostat agent to detect and communicate with the target application and use JMX to expose the Java Flight Recorder (JFR) data. You must configure the Cryostat agent to communicate with Cryostat about itself and that the agent is reachable through JMX rather than through HTTP. Prerequisites Logged in to your Cryostat web console. Created a Cryostat instance in your project. Procedure Install the Cryostat agent. For application builds that use Maven, update the application pom.xml file with the Cryostat agent JAR file information. Example pom.xml file <project> ... <repositories> <repository> <id>redhat-maven-repository</id> <url> https://maven.repository.redhat.com/earlyaccess/all/</url> </repository> </repositories> ... <build> <plugins> <plugin> <artifactId>maven-dependency-plugin</artifactId> <version>3.3.0</version> <executions> <execution> <phase>prepare-package</phase> <goals> <goal>copy</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>io.cryostat</groupId> <artifactId>cryostat-agent</artifactId> <version>0.4. X .redhat- xxxxx </version> <classifier>shaded</classifier> </artifactItem> </artifactItems> <stripVersion>true</stripVersion> </configuration> </execution> </executions> </plugin> </plugins> ... </build> ... </project> Note In the preceding example, replace 0.4. X .redhat- xxxxx with the latest version and build number of the Cryostat agent. For information about the latest version and build number of the Cryostat agent, refer to the Red Hat Maven repository . Modify your application deployment: Example apiVersion: apps/v1 kind: Deployment ... spec: ... template: ... spec: containers: - name: sample-app image: docker.io/myorg/myapp:latest env: - name: CRYOSTAT_AGENT_APP_NAME value: "myapp" # Replace this with the Kubernetes DNS record # for the Cryostat Service - name: CRYOSTAT_AGENT_BASEURI value: "http://cryostat.mynamespace.mycluster.svc:4180" - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CRYOSTAT_AGENT_CALLBACK value: "http://USD(POD_IP):9977" - name: CRYOSTAT_AGENT_AUTHORIZATION # Replace "abcd1234" with a plain-text authentication token value: "Bearer abcd1234" # This environment variable is key to the Cryostat agent # and JMX "hybrid" setup. # Set the Cryostat agent to register itself with Cryostat # as reachable through JMX, rather than reachable through HTTP. - name: CRYOSTAT_AGENT_REGISTRATION_PREFER_JMX value: "true" # Configure the application to load the agent JAR file and # to enable JMX, so that the Cryostat agent can register # itself as reachable through JMX. # To configure authentication and SSL/TLS for the JMX # connections, see <1>. - name: JAVA_OPTS value: >- -javaagent:/deployments/app/cryostat-agent-shaded.jar -Dcom.sun.management.jmxremote.port=9091 1 ports: - containerPort: 9977 protocol: TCP resources: {} restartPolicy: Always status: {} <1>: To configure authentication and SSL/TLS for the JMX connections and view further configuration options, see Configuring applications by using JMX connections . To enable Cryostat to detect the target application and connect to the Cryostat agent, configure an application Service : Example apiVersion: v1 kind: Service ... spec: ports: - name: "jfr-jmx" port: 9091 targetPort: 9091 - name: "cryostat-agent" port: 9977 targetPort: 9977 ... Additional resources Configuring applications by using the Cryostat agent Configuring applications by using JMX connections
<project> <repositories> <repository> <id>redhat-maven-repository</id> <url> https://maven.repository.redhat.com/earlyaccess/all/</url> </repository> </repositories> <build> <plugins> <plugin> <artifactId>maven-dependency-plugin</artifactId> <version>3.3.0</version> <executions> <execution> <phase>prepare-package</phase> <goals> <goal>copy</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>io.cryostat</groupId> <artifactId>cryostat-agent</artifactId> <version>0.4. X .redhat- xxxxx </version> <classifier>shaded</classifier> </artifactItem> </artifactItems> <stripVersion>true</stripVersion> </configuration> </execution> </executions> </plugin> </plugins> </build> </project> repositories { ... maven { url "https://maven.repository.redhat.com/earlyaccess/all/" credentials { username "myusername" password "mytoken" } } } plugins { id 'java' id 'application' id 'com.google.cloud.tools.jib' version '3.3.1' id 'com.ryandens.javaagent-jib' version '0.5.0' } ... dependencies { ... javaagent 'io.cryostat:cryostat-agent:0.4. X .redhat- xxxxx :shaded' COPY target/dependency/cryostat-agent-shaded.jar /deployments/app/ ENV JAVA_OPTS="-javaagent:/deployments/app/cryostat-agent-shaded.jar" docker build -t docker.io/myorg/myapp:latest -f src/main/docker/Dockerfile apiVersion: apps/v1 kind: Deployment spec: template: spec: containers: - name: sample-app image: docker.io/myorg/myapp:latest env: - name: CRYOSTAT_AGENT_APP_NAME value: "myapp" # Replace this with the Kubernetes DNS record # for the Cryostat Service - name: CRYOSTAT_AGENT_BASEURI value: "http://cryostat.mynamespace.mycluster.svc:4180" - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CRYOSTAT_AGENT_CALLBACK value: "http://USD(POD_IP):9977" 1 # Replace "abcd1234" with a plain-text authentication token - name: CRYOSTAT_AGENT_AUTHORIZATION 2 value: "Bearer abcd1234" - name: CRYOSTAT_AGENT_API_WRITES_ENABLED 3 value: true ports: - containerPort: 9977 protocol: TCP resources: {} restartPolicy: Always status: {} apiVersion: operator.cryostat.io/v1beta2 kind: Cryostat metadata: name: cryostat-sample namespace: cryostat spec: enableCertManager: true targetNamespaces: - apps oc project apps oc get secret "cryostat-ca-USD(echo -n 'cryostat/cryostat-sample' | sha256sum | cut -d ' ' -f 1)" oc set volumes deploy/my-app --add --name=cryostat-ca --secret-name=cryostat-ca-30268177e44252b3f9b7d9bf3a6db48f3a1cd3656700a6830952afc4456c0048 deployment.apps/my-app volume updated oc set volumes deploy/my-app --add --name=truststore -m /var/run/secrets/io.cryostat/truststore deployment.apps/my-app volume updated initContainers: - name: pem-to-truststore image: registry.access.redhat.com/ubi8/openjdk-11-runtime:latest command: - /bin/bash args: - -c - >- keytool -import -file /var/run/secrets/io.cryostat/cryostat-ca/tls.crt -keystore /var/run/secrets/io.cryostat/truststore/truststore.jks -trustcacerts -noprompt -storepass <my password> volumeMounts: - mountPath: /var/run/secrets/io.cryostat/cryostat-ca name: cryostat-ca - mountPath: /var/run/secrets/io.cryostat/truststore name: truststore env: - name: JAVA_OPTS_APPEND value: |- # ... -Djavax.net.ssl.trustStore=/var/run/secrets/io.cryostat/truststore/truststore.jks -Djavax.net.ssl.trustStorePassword=<my password> java -jar target/ <agent_jar_file> <pid> java -jar target/cryostat-agent-0.4.1.redhat-00001-shaded.jar 1234 java -jar target/cryostat-agent-0.4.1.redhat-00001-shaded.jar -Dcryostat.agent.baseuri= <cryostat_server_url> -Dcryostat.agent.callback= <cryostat_agent_url> -Dcryostat.agent.authorization=Bearer <authorization_value> --smartTrigger=[ProcessCpuLoad>0.2]~profile @/deployment/app/moreAgentArgs 1234 -Dcom.sun.management.jmxremote.port=<port_num> set env deployment <name> JAVA_OPTS_APPEND="..." apiVersion: v1 kind: Service spec: ports: - name: "jfr-jmx" port: 9091 targetPort: 9091 -Dcom.sun.management.jmxremote.port=<port_num> # enable JMX authentication -Dcom.sun.management.jmxremote.authenticate=true define users for JMX auth -Dcom.sun.management.jmxremote.password.file=</path/to/jmxremote.password> set permissions for JMX users -Dcom.sun.management.jmxremote.access.file=</path/to/jmxremote.access> # enable JMX SSL -Dcom.sun.management.jmxremote.ssl=true enable JMX registry SSL -Dcom.sun.management.jmxremote.registry.ssl=true set your SSL keystore -Djavax.net.ssl.keyStore=</path/to/keystore> set your SSL keystore password -Djavax.net.ssl.keyStorePassword=<password> create secret generic myapp-cert --from-file=tls.crt=/path/to/cert.pem -Dcom.sun.management.jmxremote.ssl.need.client.auth=true -Djavax.net.ssl.trustStore=</path/to/truststore> -Djavax.net.ssl.trustStorePassword=<password> set volumes deploy <myapp> --add --name=truststore --secret-name=<cryostat>-tls --sub-path=truststore.p12 --mount-path=/var/run/secrets/<myapp>/truststore.p12 set env deploy <myapp> --from='secret/<cryostat>-keystore' -Dcom.sun.management.jmxremote.ssl.need.client.auth=true -Djavax.net.ssl.trustStore=/var/run/secrets/<myapp>/truststore.p12 -Djavax.net.ssl.trustStorePassword="USD(KEYSTORE_PASS)" -Dcom.sun.management.jmxremote.port=<port_num> -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false <project> <repositories> <repository> <id>redhat-maven-repository</id> <url> https://maven.repository.redhat.com/earlyaccess/all/</url> </repository> </repositories> <build> <plugins> <plugin> <artifactId>maven-dependency-plugin</artifactId> <version>3.3.0</version> <executions> <execution> <phase>prepare-package</phase> <goals> <goal>copy</goal> </goals> <configuration> <artifactItems> <artifactItem> <groupId>io.cryostat</groupId> <artifactId>cryostat-agent</artifactId> <version>0.4. X .redhat- xxxxx </version> <classifier>shaded</classifier> </artifactItem> </artifactItems> <stripVersion>true</stripVersion> </configuration> </execution> </executions> </plugin> </plugins> </build> </project> apiVersion: apps/v1 kind: Deployment spec: template: spec: containers: - name: sample-app image: docker.io/myorg/myapp:latest env: - name: CRYOSTAT_AGENT_APP_NAME value: "myapp" # Replace this with the Kubernetes DNS record # for the Cryostat Service - name: CRYOSTAT_AGENT_BASEURI value: "http://cryostat.mynamespace.mycluster.svc:4180" - name: POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: CRYOSTAT_AGENT_CALLBACK value: "http://USD(POD_IP):9977" - name: CRYOSTAT_AGENT_AUTHORIZATION # Replace "abcd1234" with a plain-text authentication token value: "Bearer abcd1234" # This environment variable is key to the Cryostat agent # and JMX "hybrid" setup. # Set the Cryostat agent to register itself with Cryostat # as reachable through JMX, rather than reachable through HTTP. - name: CRYOSTAT_AGENT_REGISTRATION_PREFER_JMX value: "true" # Configure the application to load the agent JAR file and # to enable JMX, so that the Cryostat agent can register # itself as reachable through JMX. # To configure authentication and SSL/TLS for the JMX # connections, see <1>. - name: JAVA_OPTS value: >- -javaagent:/deployments/app/cryostat-agent-shaded.jar -Dcom.sun.management.jmxremote.port=9091 1 ports: - containerPort: 9977 protocol: TCP resources: {} restartPolicy: Always status: {} apiVersion: v1 kind: Service spec: ports: - name: "jfr-jmx" port: 9091 targetPort: 9091 - name: "cryostat-agent" port: 9977 targetPort: 9977