Is it possible to apply to a masters program without letters of recommendation?
If it is within a few years, you may be able to contact prior professors that you feel may remember you.
Otherwise, if you do not think that you can produce letters of recommendations, then I recommend you contact potential graduate schools. In fact, I believe that you should always contact potential graduate schools BEFORE applying (no matter the circumstances.) This helps you understand what they expect, prepare for potential problems, and ask any questions that you may have.
In your case, they will be able to talk with you on how to “compensate” (as you called it) for your lack of letters of recommendations. I do not know what country you are applying in, but I know that in my country (America) there are admissions counselors per each school you are applying to (for instance, religious studies admissions counselor for somebody who is majoring in Southeast Asian Religions.) If that is the case with you, then make sure you contact the school rather than the university directly to get the best possible answer.
Every school, let alone university, operates differently, and there are some schools where it may be a nonstarter to apply and then there are others that you may be able to get a waiver and/or a probation easily. Honestly and again, your best bet is to contact potential graduate schools directly.