Troubleshooting VPC flow logs with an S3 bucket using SSE-KMS encryption with CMK
Turns out I was missing one very important line in my KMS key policy:

    resources = ["*"]


Now it works fine, and my full policy looks like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Allow VPC Flow Logs to use the key",
            "Effect": "Allow",
            "Principal": {
                "Service": "delivery.logs.amazonaws.com"
            },
            "Action": [
                "kms:ReEncrypt",
                "kms:GenerateDataKey",
                "kms:Encrypt",
                "kms:DescribeKey",
                "kms:Decrypt"
            ],
            "Resource": "*"
        }
    ]
}