Sql Server - Multiple errors regarding to memory in error log
You're showing some conflicting information. 20% memory utilisation on a 4 GB server would be around 820 MB, however, your screenshots indicate somewhere between 3.8 and 4 GB is consumed by SQL Server at the moment.

Those errors are clearly indicating that SQL Server has insufficient memory available to process a request. While you could, and should, set the max server memory setting to something other than 2147483647, this won't change the fact that SQL Server wants additional memory.

You need to look at optimising the server and workload, apply best practises to the instance to ensure it is running optimally, tune your SQL code to make sure it is not using excessive memory, and you will possibly need to allocate additional memory to the server.


  After restarting the mssql-service there are no problems for something
  like 2 days and then it repeats it self.


This is because SQL Server flushes all of its caches upon restart, and it takes a while to fill that up again, given the low utilisation you've indicated. Setting a max server memory setting between 2 and 3 GB should help alleviate some of these errors because SQL Server will manage memory differently than if the setting is configured to simply consume as much memory as possible, however, it is likely that you will eventually hit these errors again as 4 GB for a Web &amp; SQL Server is quite low.


Here is a link to the SQL Server Memory Management Architecture Guide by Microsoft.
Here is a link to SQL on Linux Performance Best Practises by Microsoft.
Here is a collection of best practise tips for SQL Server that you might find useful.