How to grant all resources in an AWS account the ability to assume role?
There are several ways to enable access, and your question touches on more than one.

First method, is giving EC2 instances access to do API calls with the AWS API in another account. For this, the instances would need the appropriate IAM credentials. You can assign an IAM Role to an instance. The relationship is then "IAM Policy" -> "IAM Role" -> "IAM Instance Profile" -> "EC2 Instance". This Policy can give permission for this instance to do the sts:AssumeRole API call in the account of this IAM/EC2, and assume a role in the second account. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html

I have not tested this, but as far as I know, while the above description is correct - it will not work with the sts:AssumeRole API since it requires non-temporary (non-session) credentials. And the credentials provided to the EC2 instance are in fact sts:GetSessionToken type of credentials provided to it via the EC2 metadata service.

The same applies to worker nodes, and pods, running in EKS. There is a way to enable this for EKS that is described in details in the documentation. https://docs.aws.amazon.com/eks/latest/userguide/security-iam.html

On the other hand, if all you need is to access RDS and talk to your MySQL/PostgreSQL database - you don't need any IAM credentials. Just a clear path through the network where security groups allow these instances to access the RDS database. In EC2 and RDS, security groups allow to include foreign-account security groups as a filter for "source IP addresses". So when you choose to do something like that, you don't really need to specify the IP address of your EC2 instance where the application is executed, but rather just specify which security group is being used by that instance, with the addition of the account number. https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-security-groups.html

The S3 service does require IAM credentials to write/read data from it, but in this case it also allows to specify a permissions policy on the S3 bucket itself. So while the above method of sts:AssumeRole credentials might not work, you can create S3 buckets that do allow access from a peer account. https://aws.amazon.com/premiumsupport/knowledge-center/cross-account-access-s3/

Very similar to this, the ECR service also has an option to allow the GetLogin API call from another account. https://aws.amazon.com/premiumsupport/knowledge-center/secondary-account-access-ecr/

And naturally the simplest way, is to provide credentials directly to EC2 instances. This can be done either by writing them into a file. For example the aws CLI is using the file ~/.aws/credentials, and it allows to specify multiple named profiles so you can specify which account needs to be controlled. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

Another alternative, is to create a service that is a "Credentials ATM Machine". One excellent tool for doing this is HashiCorp Vault https://www.hashicorp.com/resources/deep-dive-vault-aws-auth-backend. AWS even has a QuickStart CloudFormation template to deploy Vault in your account. https://aws.amazon.com/quickstart/architecture/vault/