Installed Drupal 8 in English, added Hebrew, deleted English and now site is malfunctioning
My solution is comprised of part1, and part 2:

Part 1

For some reason, from the moment I deleted the English, each time I went to the pages in the site, including the one I could it bring it back in (admin/config/regional/language/add) all of these urls included an "he" (Hebrew) part. For example the add language page was:

he/admin/config/regional/language


But when I changed the he to en (English) than for some reason I could re-add the English language!

But, even thought it seemed I finished with the problem, another one came in, which I also had to solve:

URL's in the site still redirected with "he" parts even after I deleted the Hebrew language from the language lists, and thus I couldn't even flush all caches from the ui because the URL that instructs cache flushing started with an "he" part.. in the following part 2 I will describe how I solved that:

Part 2

After I re-added English, and deleted the Hebrew language from the list, I went to the "Detection and selection" tab under "Languaes" at admin/config/regional/language/detection, and unchecked there the option URL: `Language from the URL (Path prefix or domain), and than I flushed all caches with Drush (the Drupal CLI) and the problem was solved.

I than re-added Hebrew and setted it default language without any problem; Since I changed it's influence on URL's under the "Detection and selection" tab at the Languages page, I didn't have the "he"\"en" parts problem after that.