<p dir="auto">The current code for <code class="notranslate">str::not_utf8</code> condition does this:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="        cond.raise(fmt!(&quot;from_bytes: input is not UTF-8; first bad byte is %u&quot;,
                        first_bad_byte as uint))"><pre class="notranslate"><code class="notranslate">        cond.raise(fmt!("from_bytes: input is not UTF-8; first bad byte is %u",
                        first_bad_byte as uint))
</code></pre></div>
<p dir="auto">It might be more reasonable for condition handlers to recover in this situation if they had a reference to the original byte vector, rather than just a string describing what was wrong.</p>
<p dir="auto">(In one case, the byte vector being passed is a <code class="notranslate">~[u8]</code>, but in another it is <code class="notranslate">&amp;[u8]</code>; in the latter, case, attempting to do this would run into <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="12003195" data-permission-text="Title is private" data-url="https://github.com/rust-lang/rust/issues/5370" data-hovercard-type="issue" data-hovercard-url="/rust-lang/rust/issues/5370/hovercard" href="https://github.com/rust-lang/rust/issues/5370">#5370</a>, unless we were willing to allocate a copy when raising the condition, which is probably not a good idea.)</p>
<p dir="auto">(Update: blake2-ppc makes a reasonable suggestion in his comment below, though it involves a more substantial revision to the code.  I've generalized the title of this ticket accordingly.)</p>
<p dir="auto">Currently even this simple <code class="notranslate">cat</code> program:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="use io::ReaderUtil;
fn main() {
    for io::stdin().each_line |line| { io::println(line); }
}"><pre class="notranslate"><code class="notranslate">use io::ReaderUtil;
fn main() {
    for io::stdin().each_line |line| { io::println(line); }
}
</code></pre></div>
<p dir="auto">...fails on the broken or invalid UTF-8 strings (or possibly in other character encodings, as this example illustrates):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ echo 깨진 글자 | iconv -f utf-8 -t cp949 | ./test
rust: task failed at 'Assertion is_utf8(vv) failed', [...]/rust/src/libcore/str.rs:50
rust: domain main @0x7fcf32815e10 root task failed"><pre class="notranslate"><code class="notranslate">$ echo 깨진 글자 | iconv -f utf-8 -t cp949 | ./test
rust: task failed at 'Assertion is_utf8(vv) failed', [...]/rust/src/libcore/str.rs:50
rust: domain main @0x7fcf32815e10 root task failed
</code></pre></div>
<p dir="auto">...due to the byte sequence is assumed to be in UTF-8 (which is not). But there is currently no standard way to fix broken UTF-8 strings by replacing offending substrings by some other valid UTF-8, so it is hard to fix this kind of bugs.</p>
<p dir="auto">This issue is ultimately linked to the general character encoding handling (libiconv binding, perhaps?) and a strict distinction between byte sequence and Unicode (UTF-8) string. I found Python's approach reasonable (bytes and str are separated, converted to each other via <code class="notranslate">encode</code> and <code class="notranslate">decode</code> methods, normal file <code class="notranslate">open</code> reads bytes, <code class="notranslate">codecs.open</code> with an encoding converts them to str), but I'm really not sure about the actual interface.</p>