How to start building your own DBMS (based on PostgreSQL)?
My advice is to get involved with an open source database project. I would go with PostgreSQL, but there are of course alternatives.

If you go with PostgreSQL:


Use the software. It is impossible to learn about databases without actually using them.
Build PostgreSQL from source.
Subscribe to the mailing lists and read them. There you can read what knowledgeable people have to say, and one can learn a lot from that.
Whenever you run into a problem or don't understand the workings of something, dig into the source. If you get stuck, ask for help.
Read this article.
Tune in on the pgsql-hackers mailing list.
Start reviewing patches and contributing.


Keep your eyes open for the different ways that other databases do it. After a few years, you will have gained some understanding. Then you may be in a position to come up with your own ideas on architecture.