title: Kubernetes Job failed with no logs, no termination reason, no events
body: <p>I ran a Job in Kubernetes overnight. When I check it in the morning, it had failed. Normally, I'd check the pod logs or the events to determine why. However, the pod was deleted and there are no events.</p>
  
  <pre><code>kubectl describe job topics-etl --namespace dnc
  </code></pre>
  
  <p>Here is the <code>describe</code> output:</p>
  
  <pre><code>Name:           topics-etl
  Namespace:      dnc
  Selector:       controller-uid=391cb7e5-b5a0-11e9-a905-0697dd320292
  Labels:         controller-uid=391cb7e5-b5a0-11e9-a905-0697dd320292
                  job-name=topics-etl
  Annotations:    kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{},"name":"topics-etl","namespace":"dnc"},"spec":{"template":{"spec":{"con...
  Parallelism:    1
  Completions:    1
  Start Time:     Fri, 02 Aug 2019 22:38:56 -0500
  Pods Statuses:  0 Running / 0 Succeeded / 1 Failed
  Pod Template:
    Labels:  controller-uid=391cb7e5-b5a0-11e9-a905-0697dd320292
             job-name=topics-etl
    Containers:
     docsund-etl:
      Image:      acarl005/docsund-topics-api:0.1.4
      Port:       &lt;none&gt;
      Host Port:  &lt;none&gt;
      Command:
        ./create-topic-data
      Requests:
        cpu:     1
        memory:  1Gi
      Environment:
        AWS_ACCESS_KEY_ID:      &lt;set to the key 'access_key_id' in secret 'aws-secrets'&gt;      Optional: false
        AWS_SECRET_ACCESS_KEY:  &lt;set to the key 'secret_access_key' in secret 'aws-secrets'&gt;  Optional: false
        AWS_S3_CSV_PATH:        &lt;set to the key 's3_csv_path' in secret 'aws-secrets'&gt;        Optional: false
      Mounts:
        /app/state from topics-volume (rw)
    Volumes:
     topics-volume:
      Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
      ClaimName:  topics-volume-claim
      ReadOnly:   false
  Events:         &lt;none&gt;
  </code></pre>
  
  <p>Here is the job config yaml. It has <code>restartPolicy: OnFailure</code>, but it never restarted. I also have no TTL set so pods should never get cleaned up.</p>
  
  <pre><code>apiVersion: batch/v1
  kind: Job
  metadata:
    name: topics-etl
  spec:
    template:
      spec:
        restartPolicy: OnFailure
        containers:
          - name: docsund-etl
            image: acarl005/docsund-topics-api:0.1.6
            command: ["./create-topic-data"]
            env:
              - name: AWS_ACCESS_KEY_ID
                valueFrom:
                  secretKeyRef:
                    name: aws-secrets
                    key: access_key_id
              - name: AWS_SECRET_ACCESS_KEY
                valueFrom:
                  secretKeyRef:
                    name: aws-secrets
                    key: secret_access_key
              - name: AWS_S3_CSV_PATH
                valueFrom:
                  secretKeyRef:
                    name: aws-secrets
                    key: s3_csv_path
            resources:
              requests:
                cpu: 1
                memory: 1Gi
            volumeMounts:
              - name: topics-volume
                mountPath: /app/state
        volumes:
          - name: topics-volume
            persistentVolumeClaim:
              claimName: topics-volume-claim
  </code></pre>
  
  <p><strong>How can I debug this?</strong></p>