Do I have to worry that root user can login into MariaDB 10.5 without password I remember when installing MySQL Server 8.0, or in some database setups, there are warnings about no password login. Should I worry about this? I think if someone gets access to root user on my debian they can reset root password of MariaDB. What is the real answer here? I appreciate doc reference.
Only if you have only localhost access.
When  the attacker has somehow direct baccess to your server, it isn't that much important, but will still remain a nuisance for the hacker.
When you let access from outside the computer, it is vital that there is a strong password, which lets casual users not gain full access to the hole database.
Overall, yes make a passowrd