Cheap alternative to Azure SQL database
You may be describing a key value store. It is unlikely that you need SQL or relational.
To avoid vendor-lock-in, and to control your costs. Avoid complex services from the cloud providers. Rent a machine or virtual machine from them, and put your own services on it. There are many Free and Open Source databases. Do some research: start with key-value stores (these are the simplest, fastest, but also low functionality, databases).