MariaDB keeps crashing after a few hours with ridiculous memory expectations Mariadb keeps crashing with typical errors from /var/log/mysql/error.log being:

... terribly wrong ...
stack_bottom = 0x0 thread_stack 0x20000
(my_addr_resolve failure: fork)
...
2019-01-27  9:04:30 3069758672 [ERROR] mysqld: Out of memory (Needed 130760704 bytes)
2019-01-27  9:04:30 3069758672 [ERROR] mysqld: Out of memory (Needed 98058240 bytes)
2019-01-27  9:04:30 3069758672 [ERROR] mysqld: Out of memory (Needed 73531392 bytes)
2019-01-27  9:04:30 3069758672 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-01-27  9:04:30 3069758672 [Note] InnoDB: The InnoDB memory heap is disabled
2019-01-27  9:04:30 3069758672 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-01-27  9:04:30 3069758672 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-01-27  9:04:30 3069758672 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-01-27  9:04:30 3069758672 [Note] InnoDB: Using Linux native AIO
2019-01-27  9:04:30 3069758672 [Note] InnoDB: Using generic crc32 instructions
2019-01-27  9:04:30 3069758672 [Note] InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(138543104 bytes) failed; errno 12
2019-01-27  9:04:30 3069758672 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
2019-01-27  9:04:30 3069758672 [ERROR] Plugin 'InnoDB' init function returned error.
2019-01-27  9:04:30 3069758672 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-01-27  9:04:30 3069758672 [Note] Plugin 'FEEDBACK' is disabled.
2019-01-27  9:04:30 3069758672 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-01-27  9:04:30 3069758672 [ERROR] Aborting


My platform is a Raspberry Pi Zero with 512M of RAM and running 2018-10-09-raspbian-stretch.  I’ve installed Mariadb version
mysql  Ver 15.1 Distrib 10.1.23-MariaDB, for debian-linux-gnueabihf (armv7l) using readline 5.2

From reading various forums I’ve changed the following:

/etc/mysql/mariadb.conf.d/50-server.cfg:
change key_buffer_size to 64K
change max_connections to 10
change query_cache_limit to 16K
change query_cache_size to 128K


I tried to add 

performance_schema=0


to /etc/mysql/my.cnf but Mariadb didn’t recognize the variable.

There are only two clients to the database – an Apache2 server on the same device which connects every 5 seconds and a daemon I wrote in C which connects every second, both for a brief time.  I’ve checked my code and each of them closes the connection to Mariadb correctly.

Any help would be very much appreciated.

EDIT: After enabling Slow Query Logging, I find that most queries take around 1ms-3ms to run, but in the lead-up to the crash, this time began to ramp up to 1.8 seconds, before the final log event of:

SET timestamp=1548626761;
SELECT * FROM timers;
# Time: 190128  6:06:03
# User@Host: pi[pi] @ localhost []
# Thread_id: 4  Schema: hptimer  QC_hit: No
# Query_time: 1.865374  Lock_time: 0.801896  Rows_sent: 0  Rows_examined: 8
# Rows_affected: 1
SET timestamp=1548626763;
UPDATE relays SET state=0 WHERE id=3;
/usr/sbin/mysqld, Version: 10.1.23-MariaDB-9+deb9u1 (Raspbian 9.0). started with:
Tcp port: 3306  Unix socket: /var/run/mysqld/mysqld.sock
Time                 Id Command    Argument


then no further information.

EDIT#2: Thank you for the suggestions to monitor memory usage/other processes.  I found my C daemon process was consuming an additional 100KiB per call using "top", hence the OOM.  I shall pull out my K&amp;R and take a closer look at my code before blaming other applications again :)

Best regards,
Peter

EDIT#3: You must always use 

mysql_free_result( res );


after a query in C after your query has been processed.

https://dev.mysql.com/doc/refman/8.0/en/mysql-free-result.html
You must always use

mysql_free_result( res );
after a query in C after your query has been processed.

https://dev.mysql.com/doc/refman/8.0/en/mysql-free-result.html

This solved my OOM and memory leak from my C daemon, which was the cause of the problem.