title: Using the Beam Python SDK and PortableRunner to connect to Kafka with SSL
body: <p>I have the code below for connecting to kafka using the <a href="https://beam.apache.org/documentation/sdks/python/" rel="nofollow noreferrer">python beam sdk</a>. I know that the <code>ReadFromKafka</code> transform is run in a java sdk harness (docker container) but I have not been able to figure out how to make <code>ssl.truststore.location</code> and <code>ssl.keystore.location</code> accesible inside the sdk harness' docker environment. The <code>job_endpoint</code> argument is pointing to <code>java -jar beam-runners-flink-1.10-job-server-2.27.0.jar --flink-master localhost:8081</code></p>
  <pre><code>pipeline_args.extend([
      '--job_name=paul_test',
      '--runner=PortableRunner',
      '--sdk_location=container',
      '--job_endpoint=localhost:8099',
      '--streaming',
      &quot;--environment_type=DOCKER&quot;,
      f&quot;--sdk_harness_container_image_overrides=.*java.*,{my_beam_sdk_docker_image}:{my_beam_docker_tag}&quot;,
  ])
  
  with beam.Pipeline(options=PipelineOptions(pipeline_args)) as pipeline:
      kafka = pipeline | ReadFromKafka(
          consumer_config={
              &quot;bootstrap.servers&quot;: &quot;bootstrap-server:17032&quot;,
              &quot;security.protocol&quot;: &quot;SSL&quot;,
              &quot;ssl.truststore.location&quot;: &quot;/opt/keys/client.truststore.jks&quot;, # how do I make this available to the Java SDK harness 
              &quot;ssl.truststore.password&quot;: &quot;password&quot;,
              &quot;ssl.keystore.type&quot;: &quot;PKCS12&quot;,
              &quot;ssl.keystore.location&quot;: &quot;/opt/keys/client.keystore.p12&quot;, # how do I make this available to the Java SDK harness 
              &quot;ssl.keystore.password&quot;: &quot;password&quot;,
              &quot;group.id&quot;: &quot;group&quot;,
              &quot;basic.auth.credentials.source&quot;: &quot;USER_INFO&quot;,
              &quot;schema.registry.basic.auth.user.info&quot;: &quot;user:password&quot;
          },
          topics=[&quot;topic&quot;],
          max_num_records=2,
          # expansion_service=&quot;localhost:56938&quot;
      )
  
      kafka | beam.Map(lambda x: print(x))
  </code></pre>
  <p>I tried specifying the <a href="https://github.com/apache/beam/blob/681ea6fe7e6adf18fcf848b69417b3daf536730a/sdks/python/apache_beam/options/pipeline_options.py#L824-L833" rel="nofollow noreferrer">image override</a> option as <code>--sdk_harness_container_image_overrides='.*java.*,beam_java_sdk:latest'</code> - where <code>beam_java_sdk:latest</code> is a docker image I based on <code>apache/beam_java11_sdk:2.27.0</code> and that pulls the credetials in its entrypoint.sh. But Beam does not appear to use it, I see</p>
  <pre><code>INFO  org.apache.beam.runners.fnexecution.environment.DockerEnvironmentFactory  - Still waiting for startup of environment apache/beam_java11_sdk:2.27.0 for worker id 1-1
  </code></pre>
  <p>in the logs. Which is soon inevitebly followed by</p>
  <pre><code>Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: Failed to load SSL keystore /opt/keys/client.keystore.p12 of type PKCS12
  </code></pre>
  <p>In conclusion, <em><strong>my question is this, In Apache Beam, is it possible to make files available inside java sdk harness docker container from the python beam sdk? If so, how might it be done?</strong></em></p>
  <p>Many thanks.</p>