Title: Encoding and byte errors in Ruby using OpenSSL?
Tags: javascript;ruby-on-rails;ruby;encryption;openssl
Question: I was getting this error:
incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string)
when trying to pass a variable from Rails to JavaScript via this line:
window.nonce = '<%= j @nonce %&gt;';
My @nonce variable is an OpenSSL RSA public key encryption created by:
@nonce = Rails.rsaEncUsingPublic(pubKey, randomStr)
def self.rsaEncUsingPublic(key, msg)
return key.public_encrypt msg
I then tried adding force_encoding("UTF-8") to the end of the rsaEncUsingPublic function which changed the error to:
invalid byte sequence in UTF-8
Now I clearly don't want to strip characters from an encrypted variable, however the same encryption function works fine everywhere else, until I'm passing it to JavaScript.
Printing out @nonce (with or without force encoding) does give a lot of gibberish:
#??7:A}p[?ͼg/?%ŋ??=????h9?jg?    W?V?j ?}??G???t?7?i?:"???"#HM?0?L?????*r:ɦYaB&amp;v&amp;?5mǓŌ,???[U?Dt??G???Tև?&amp;~??6f???????P??<GKV?`? p?K??B???????[?yj6?=?;?
???p?j=?Z?? ?[?5i??i?t?!???^?L{;??,??Ma\_mg?|??]A?????"??X:??;???      ?y??\纘???#    ]?M" ?
?N
@nonce.encoding prints out UTF-8.
@nonce.inspect pints out:
"\u0015\xC0jn\xE7\xBC\xE4\u0016gV\x84&amp;-ˌ+ŚA:4\xB1(\xC0\xEAv\x91\xE8&gt;\u001D\x92ږ\xF6\xDC\xEE\x9A)\xC7&amp;O\u001A\x90fเ\e\x9Bb*\xF2\xE2\u001E\xB9V\x9E\xBB\x9AUЕcU\u001E~\u0011\u0001$մ\xF8J\xED\xFE^\"\u001EC\xBD8\u0002\xBA\xDC\xDFIЊ, KU\u0000\u0014\u0015\x92_w\x95\x89\xD0-OfG\xB5\xF8LC\x9BO\\0j<ƥ\xA5\u001Dw(t?\xA4\xA2\u00174\xB5Š\xE3\x91s\xDA\u0002i\xB3\u0003Q\u000F\xF4\xDB5\x80\xD8\xE0./\x8B\x8A߳0\u0001\x91=$T\xCB\bLh\xF3\u001C\xFD\xBF\x95I%=gQ\u000F}\x8F_w\xFAn\x90\x81\xFC\b4\x9E\xC1\xD7y\xBC\xE8\xA4cQY\xB2@s1\xD7\xC9+\xA7\xEA&gt;\xA5\xBC\xCF\xC81:TG\xFD\x88\xCCS\x90\xB1\x9Cv\xA3ݘ,\xA1;\xA5\xEE\xE4q9\u0000w\xB9\xB3\u0014\xD9\u0015\x8B\x82nw\ej\x82xkm)\x9Aa\xF1\xDD۬\xA2"
All help would be appreciated!
Comment: Could you paste what is stored within `@nonce`. Also, what is `@nonce.encoding`?

Comment: Try also add `@nonce.inspect`. Also what do you get with `@once.encoding`?

Comment: Edited the question @BroiSatse

Comment: update again @BroiSatse

Here is the accepted answer: So, the reason it is happening is within j method, exactly in this line:
result = javascript.gsub(/(\|<\/|\r\n|\342\200\250|\342\200\251|[\n\r"'])/u) {|match| JS_ESCAPE_MAP[match] }
Note the lovely u at the end of the regex. This forced Ruby to fail the match if the given string is not UTF-8 compatible.
Now, when you are encoding a string the result is not meant to be any form of human readable string - it is purely binary and no encoding will display anything useful. Ruby, by default, is trying to display it with 'UTF-8', but this string is not 'UTF-8' compatible as it may contain non-UTF-8 sequences (since it may contain literally anything).
Now the solution won't be easy as this string is not a valid JavaScript jenniferkrueger@example.net. Also, even if you managed to convert it so that JavaScript, JavaScript might save the string representation differently, and the important thing is to keep the binary information unchanged. The easiest way is probably to convert the encrypted string into a number:
window.nonce = '<%= @nonce.unpack('B*').first.to_i(2) %&gt;';
It will result in a number which has a binary representation that is exactly the same as for the encrypted string. You only need to be sure it is handled properly everywhere.
Question: how is this string to be used in JavaScript?
Comment for this answer: Ah, missed first. Updated.
Comment for this answer: @roly - note there is no `j` call above.
Comment for this answer: Thank you! the string will be decrypted using the private key that is stored with the client (uses the forge.pki library in JS) I've already handled the key distrubution so the public/private keys and encryption/decryption between the 2 systems (js and ruby) all work fine, its just this passing of the encrypted string thats proving to be a problem. Using your suggestion, how would I then go about using that in the JS? Thank you!
Comment for this answer: Also note that using your suggestion, still gives an error on that line of `undefined method to_i' for #`
Comment for this answer: `undefined method 'gsub' for #` basically, I think i'm stuffed aha!
Comment for this answer: my bad! Thank you, that works with the number representation, now to figure out how to work with that
Comment for this answer: https://github.com/digitalbazaar/forge has a method called `forge.util.hexToBytes(hex);` have you any idea if doing something like this would work? I tried doing `.unpack('H*')` and running the hextobytes on the window.nonce, but no idea if I'm doing that correctly


{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 1, 'CommentCount': 4, 'ContentLicense': 'CC BY-SA 3.0', 'CreationDate': '2014:00:03/10/14', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<javascript><ruby-on-rails><ruby><encryption><openssl>', 'Title': 'Encoding and byte errors in Ruby using OpenSSL?', 'ViewCount': 216, 'id': 22299019, 'user_AboutMe': '', 'user_AccountId': 1467940.0, 'user_DisplayName': 'rogy', 'user_Reputation': 430.0, 'user_Views': 58.0, 'user_id': 1380276.0}}