How to stand out in mathematics PhD admissions as a student from an online school, with no research or opportunities to get strong recommendations?
tl;dr
Get a MSc from a top Israeli university first. It will practically ensure that you will get accepted to just about any PhD program that you want to. And a math MSc from a top Israeli uni is a pretty good thing to have. (edit see below)



I'm an Israeli currently studying outside of Israel, so I might give you a more relevant perspective on the issue. Although I'm in earth sciences, and not math, I think you will find this useful.

I am having a hard time understanding some of your circumstances. You claim that you will graduate in a year and a half. Assuming you are studying full-time, you are only halfway through and it may be too early to start thinking about your PhD, let alone fly to institutions in the world and meet with them (unless you happen to be there as a tourist).

You are correct that graduating from the Israeli Open University is hardly beneficial in terms of reputation and prestige. It is also close to impossible to get meaningful and honest recommendation letters. I do not know how much people trust GPA scores from a distance learning institution, and this may also hurt your chances. Unfortunately, as hardly any research is done at OU, I doubt that you will find a research partner to work on a publication. Even if your numbers are right, the fact that you do not study in a university where you have day-to-day interaction with other students and senior stuff is not going to help the admissions committees to choose you over other candidates.

Instead of thinking about PhD research abroad, I suggest you start with a MSc degree back home in Israel. There are several reasons for that:


Math degrees in Israeli universities have excellent reputation (particularly Tel Aviv, HUJI and Technion). This alone might boost your chances.
You are still an undergrad, halfway through. You have not been exposed to research yet. If you were in a physical university, you might be exposed to research around you, which is something. Yet you don't even have that. You do not know if research is what you want to do. Who knows, you might get offered a job and leave academia. Going for MSc will be your "introduction" to research and will give you a chance to get your desired publication, if you wish to pursue this further.
Starting a PhD program with MSc background will make you a much better student. I can sometimes see the difference between PhD students with and without prior MSc experience in the institution I am currently at.
Admission committees, in general, look favourably upon applicants with proved research experience in the form of MSc.


I will give you an example from my own experience. Having graduated my BSc with a research component, I was eligible for PhD studies abroad without having to get MSc. However, I did study for MSc in Israel, mostly for the reasons above. It made getting accepted to my institution of choice much easier, I had more than one publication by the time I applied for the PhD, it gave me a different perspective on what research is and what is it that I want to do, or more importantly, do not want to do.


EDIT

OP mentioned that conducting research in Israel is not possible due to personal reasons. I don't see how this sits with OP's idea of taking MSc research in Israel, but here are two more ideas:


Go for a one year Honours degree in countries that have it (for example, UK or Australia). This is something that's usually done straight after BSc and is one year of research.
Go for MSc in a country where doing this is common as a precursor to PhD research (continental Europe mostly, also Israel btw).


These two are very expensive solutions because you can't usually get funded or a scholarship for this, unlike doing PhD research. I would personally go for number two from the list above.

Again, getting straight to PhD after finishing your BSc in the Open University is almost impossible. If you do get it, it's probably in a low ranked institution and that might not be the best thing for your career. Regardless of whether you are accepted or not, I still think that you should have some kind of research experience before going for a PhD, particularly as you're done your education so far in a research-free environment.