AWS Production Configuration ASP.NET Core & Docker
If you are going full on AWS, I would suggest to have a look at AWS SSM parameters. When EC2 or ECS container are starting, you have a script that is going to fetch the parameters. Then you can do whatever you need to do: replace variables in config files with the parameters fetched previously for example.
You could also use files stored in S3. You would be able to copy it and place it in the right application folder when your EC2 or ECS container are starting.