How does one test the freewheel?
You are correct about the two failure modes.  Try lubricating your freewheel with a light oil.  If it is dry or dirty that can cause it to not engage.  If that does not fix it then you will need to replace it.