RDS DB migrations using ECS and Fargate
If you already have the script/tool in place to execute the database migrations then what you might need is to run the tool/script once. AWS ECS allows to schedule the executions of tasks (containers), you can set it up in a way that the container can run just only once.

These ECS tasks can be scheduled via a cron-like job setup or on a CloudWatch event response:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/scheduled_tasks.html