Is it "okay" to submit to a workshop without proceedings while in review for publication?
Short answer:  okay, and a quite good idea, for the reasons and with few limits, see below.


  in academia one should never submit the same paper to two journals


If only noone had done that... 

So I suppose that you have a journal paper under review (your title). And the  workshop asks for a contribution. I doubt you will directly send the paper under review to the workshop committee. So you will write something "else" for the workshop, possibly a shortened version. 

Recently, I read a chronic in IEEE Information Theory Society Newsletter (see below for the full quote). In the 1950s, apparently, you could present a paper to the associated workshop/conference (only) when it was accepted or published before in the journal. This has the advantage the audience could read the paper in detail before the talk, and allow nicer discussions.

Recently, some IEEE conferences (ICASSP, ICIP, GlobalSIP) have begun the accept "submissions", after the official deadlines, from papers already accepted in IEEE Signal Processing Letters (which has the same 5-page same format as the conference). The paper, once accepted, is presented at the conference, but it is not in the abstract book: of course, it is already "being published" by the journal. 

These examples show that limiting the clutter of unfinished works, submitted too early and several times at conferences, was and still is possible. Indeed, if one writes a conference paper, after presenting it, it is sometimes difficult to find the energy to write the full paper. 

So this sounds perfectly fine to me. The main thing to care about is ownership: as the preprint (submitted) is still yours, and as long as it does not infringe the workshop rules. 

I believe though it is honest, and useful, to mention in your workshop absract, that the extended version of the work is "under submission", "under revision" in a journal (you can mention the name or not), or accepted with the date of acceptance.



IEEE Information Theory Society Newsletter, Vol. 61, No. 2, June 2011 (pdf), in the The Historian’s Column:


  It is worth noting in Fano’s letter that the number of sessions was
  six (6!) and all of them were in series. Each paper was presented for
  about 20 minutes with 10 minutes left for discussion which was
  invariably lively and had to be stopped by the chair person to make
  sure that the schedule was kept in time. But consider this: The papers
  were FIRST published in the Transactions and THEN they were presented
  at the Symposium!! That is a total reversal of what we do today. The
  distinct advantage of this arrangement was that the audience had the
  benefit of studying the papers carefully ahead of time, which enabled
  an in-depth discussion after the presentation. Not a bad idea!