<p dir="auto">I work with a byte stream (from Azure DataLake <a href="https://pypi.python.org/pypi/azure-datalake-store/0.0.19" rel="nofollow">https://pypi.python.org/pypi/azure-datalake-store/0.0.19</a> which only supports byte stream) that has a UTF-8 byte order mark, and want to read it into a data frame.</p>
<p dir="auto">pandas.read_json fails.<br>
For comparison, pd.read_csv(file, lines=True, encoding='utf-8-sig') works fine with a similar file</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import pandas as pd

def skip_utf_8_bom(file):
    bom = file.read(3)
    # print(bom)
    if bom != b'\xef\xbb\xbf': # undo read
        file.seek(len(bom), 1) 

path = 'sample-utf-8-sig.txt'

#works
with open(path, 'rb') as file:   
    skip_utf_8_bom(file)
    df = pd.read_json(file, lines=True, encoding='utf-8-sig')    
df

#fails
with open(path, 'rb') as file: 
    df = pd.read_json(file, lines=True, encoding='utf-8-sig')    
df
"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-k">def</span> <span class="pl-en">skip_utf_8_bom</span>(<span class="pl-s1">file</span>):
    <span class="pl-s1">bom</span> <span class="pl-c1">=</span> <span class="pl-s1">file</span>.<span class="pl-en">read</span>(<span class="pl-c1">3</span>)
    <span class="pl-c"># print(bom)</span>
    <span class="pl-k">if</span> <span class="pl-s1">bom</span> <span class="pl-c1">!=</span> <span class="pl-s">b'<span class="pl-cce">\xef</span><span class="pl-cce">\xbb</span><span class="pl-cce">\xbf</span>'</span>: <span class="pl-c"># undo read</span>
        <span class="pl-s1">file</span>.<span class="pl-en">seek</span>(<span class="pl-en">len</span>(<span class="pl-s1">bom</span>), <span class="pl-c1">1</span>) 

<span class="pl-s1">path</span> <span class="pl-c1">=</span> <span class="pl-s">'sample-utf-8-sig.txt'</span>

<span class="pl-c">#works</span>
<span class="pl-k">with</span> <span class="pl-en">open</span>(<span class="pl-s1">path</span>, <span class="pl-s">'rb'</span>) <span class="pl-k">as</span> <span class="pl-s1">file</span>:   
    <span class="pl-en">skip_utf_8_bom</span>(<span class="pl-s1">file</span>)
    <span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-en">read_json</span>(<span class="pl-s1">file</span>, <span class="pl-s1">lines</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, <span class="pl-s1">encoding</span><span class="pl-c1">=</span><span class="pl-s">'utf-8-sig'</span>)    
<span class="pl-s1">df</span>

<span class="pl-c">#fails</span>
<span class="pl-k">with</span> <span class="pl-en">open</span>(<span class="pl-s1">path</span>, <span class="pl-s">'rb'</span>) <span class="pl-k">as</span> <span class="pl-s1">file</span>: 
    <span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-en">read_json</span>(<span class="pl-s1">file</span>, <span class="pl-s1">lines</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, <span class="pl-s1">encoding</span><span class="pl-c1">=</span><span class="pl-s">'utf-8-sig'</span>)    
<span class="pl-s1">df</span></pre></div>
<h4 dir="auto">Problem description</h4>
<p dir="auto">pd.read_json seems not to be able to process the  encoding='utf-8-sig' parameter.<br>
Expected behavior is that it allows to work with byte streams with an utf-8 byte order mark</p>
<p dir="auto"><a href="https://github.com/pandas-dev/pandas/files/1873467/sample-utf-8-sig.txt">sample-utf-8-sig.txt</a></p>
<p dir="auto">xref <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="158265190" data-permission-text="Title is private" data-url="https://github.com/pandas-dev/pandas/issues/13351" data-hovercard-type="pull_request" data-hovercard-url="/pandas-dev/pandas/pull/13351/hovercard" href="https://github.com/pandas-dev/pandas/pull/13351">#13351</a></p>
<p dir="auto"><a href="https://github.com/pydata/pandas/commit/964b7bba7f5878c79130479f75461c58dd0c4b3e">Here's the test</a>, currently skipped</p>