Are there any open access databases of all (or close enough) academic papers ever published? (not full-text)
I suppose you mean databases that are free to query so that you get to see all metadata stored without any subscription (i.e., not Scopus or Web of Science)?
Yes, there are various free platforms of that kind. My favourite (cross-disciplinary) ones are:

LENS.org (my favourite one - very comprehensive, and allows additional analyses and visualisations)
Microsoft Academic (not bad, powered by a semantic search approach)
Google Scholar (probably the most prominent one)
Semantic Scholar (another semantic search approach)
ScienceOpen (not too familiar with it)
CrossRef Search (only offers metadata about articles that have a DOI registered at CrossRef)
Wizdom.ai (offers interesting analyses, e.g. regarding research trends and grants)