'Out of memory' Error for MariaDB on RDS
Usually the quick fix is to decrease the biggest memory usage -- which should be innodb_buffer_pool_size.  What is that currently set to?  Do you have other applications running on the same VM?  How much room are they taking?

On a 4GB 'machine', I don't recomment innodb_buffer_pool_size be higher than 1500M, but less if the server is not dedicated to MySQL.