Toilet water tank sometimes takes hours to fill and sometimes it fill up right away
Your problem could be a flapper valve that does not always seat down tight on the flush drain at the bottom of the toilet water tank. This can cause a small flow of water to continue to drain from the tank as it attempts to fill. Sometimes the flapper will plop down in the correct position and allow the tank to fill normally. 

In the leaking case, as the tank does start to slowly fill, the water pressure from above will try to push the flapper valve to a fully closed position thus starting to close off the leak and eventually allowing the tank to fill up. 

Flapper valves, despite the claims from the manufacturer, can react with the water and become either hardened or start to decompose to a rubbery gunk. (Even with the best ones I can purchase I need to replace mine about once every one to two years). Both of these cases can lead to the flapper valve seating problem like I describe above. The fix is to purchase a new one. Typical install is easy and usually takes all of 5 minutes. There may be some extra time to adjust the chain from the flush lever to get it operate correctly. Chain too long and it can get under the flapper and create a leak and too short and the lever tugs on the flapper putting undue stress on the part of the flapper where the chain attaches.