Using Structured Filetypes as Databases
I would have to say, resoundingly, that your co-worker is correct and you are not. dBASE (.dbf) is a great example of a flat file database--which has been used for 35 years or so. https://en.wikipedia.org/wiki/.dbf

Even RDBMS like SQL use simple 8kb pages for storing data. The tools we use to access the data makes it look more complex, but in reality it's stored in a very simple structure. Using hints, you can even have SSMS display these pages of data right in the output window. 

By going by the simple definition of the database, it's a structured collection of data. CSV, XML, JSON, DBF all meet this definition.