How to keep and organize a research journal/log/diary/notebook?
Be aware that only you will ultimately be able to answer this question! Each individual works differently, so take the following cum grano salis.

As you've noted, a physical notebook does not always work well. Not only it is difficult to carry and easy to lose, it is also not easily searchable. (Despite this, I personally use a series of cheap notebooks and paper folders to enhance my thinking and contain random notes during the initial stages of research.) For many researchers, electronic files are the best alternative to paper. Your LaTeX file is a good beginning, and sounds like all you really need is a way to organize the huge mass of material now contained in a single file. 

I would suggest breaking the file into several files, each containing a separate aspect of your current journal. How you organize this is ultimately up to you. Use the logic and structure that meshes with the way your brain works. Only one hard and fast rule: Never have a file labeled miscellaneous! This will end up catching all sorts of odds and ends and junk, and you will NOT be able to remember what the file actually contains.