How does an encrypted DynamoDB (the same question applies to RDS as well) query work?
For DynamoDB encryption at rest, it means the table is encrypted on disk, not each of the attributes of the items in the table. This is true for any database talking about encryption at rest. That at rest means on disk. Now if you were to implement application data level encryption on your own, then what you are talking about would come into play.

One thing to know is that as of a few weeks ago, all DynamoDB tables are encrypted at rest by default.