MySQL - Flush Memory after loading files I am loading a great amount of data which are in CSV files using LOAD FILE command. It is working fine and after loading that files, it executes some data transformation...

The problem is that, after loading for example 12 files (400 MB each one), mysql process consumes almost all memory available, it is not flushing, and so, the machine slow down, and I need to restart mysql to continue to load data files.

MySQL 8.0.17

Any ideas how to solve this issue?


Added Store Procedure

BEGIN
DECLARE sequenceid INT;

DECLARE `_rollback` BOOL DEFAULT 0;

DECLARE CONTINUE HANDLER FOR SQLEXCEPTION SET `_rollback` = 1;


START TRANSACTION;

INSERT INTO filenamedt (filename, filedate, fileinitialid) VALUES (CAfilename, now(), (SELECT MAX(imp_sequence_id) + 1 FROM imp_siscori_sequence));

INSERT INTO imp (select imp, ordem, CONCAT(TRIM(data),'01')) FROM temp_imp);

INSERT INTO imp_data (imp_id, unid_comerc, incoterm) (SELECT imp_id, TRIM(unidadeComercial),  TRIM(incoterm),  TRIM(natInformacao) FROM temp_imp );

INSERT INTO imp_description (imp_id, imp_description) (SELECT imp_id, TRIM(descricaoProduto) FROM temp_imp);

UPDATE imp_sequence SET imp_sequence_id  = (SELECT MAX(imp_id) + 1 FROM temp_imp_siscori);

TRUNCATE TABLE temp_imp;

SET @m = (SELECT MAX(imp_sequence_id) + 1 FROM imp_sequence); 

SET @s = CONCAT('ALTER TABLE temp_imp AUTO_INCREMENT=', @m);

PREPARE stmt1 FROM @s;

EXECUTE stmt1;

DEALLOCATE PREPARE stmt1;

IF `_rollback` THEN

    ROLLBACK;

ELSE

    COMMIT;

END IF;
If you are not using MyISAM for anything (and you should not), then change key_buffer_size 999997440 to only 10M.  That is a wasted gigabyte.

Also, this seems quite large for your 8GB of RAM:  tmp_table_size  268435456.  Lower it to 16M.