Unexpected HY001: Memory allocation error after MySQL upgrade
And you have large JSON or Geometry columns?  (Always provide SHOW CREATE TABLE when asking a question.)
Either increase or decrease sort_buffer_size and innodb_sort_buffer_size.
https://bugs.mysql.com/bug.php?id=103465
and
https://bugs.mysql.com/bug.php?id=103225