What is required for transatlantic flight in a private plane?
While probably legal to fly over the Atlantic VFR, it appears getting out of Canadas airspace is a different matter. After a bit of googling I found this article on COPA's website, saying:


  Both for flying in the northern parts of Canada as well as in
  Greenland airspace, an ADF is legally required (refer to TC AIM
  3.16.10 and CAR 605.18). I didn’t have one, so one had to be installed. Other than that, FDOW didn’t need any modifications. IFR
  approved GPS is an absolute must; we had three GPS receivers on board.
  
  Speaking of IFR: As per CAR 602.19, pilots attempting trans-oceanic
  flight must hold an instrument rating. Even though an IFR flight plan
  is only required above 5,500 feet across the Atlantic, NavCanada will
  not issue an oceanic VFR clearance when leaving Canada. In addition,
  it is extremely unlikely for any airplane to stay in VMC during the
  whole trans-Atlantic flight. Therefore, it’s mandatory that both,
  pilot and airplane are certified and current IFR.


That being said, weather over the Atlantic can be very treacherous, especially if you're unable to climb on top.

There are quite a few stories around about crossing the Atlantic in smaller airplanes, there's also an Ice Pilots NWT episode (season 1 episode 9) where they ferry two water-bombers to Turkey, although not very technical, it's a nice watch.