<p dir="auto"><em>Original ticket <a href="http://projects.scipy.org/numpy/ticket/1445" rel="nofollow">http://projects.scipy.org/numpy/ticket/1445</a> on 2010-04-01 by trac user Onanymous, assigned to unknown.</em></p>
<p dir="auto">Just like it is implemented in gnuplot, different datasets are separated by two empty lines, when reading the file I can choose which dataset to read.<br>
Probably another parameter, like loadtxt(fname, dataset=0).<br>
Meanwhile there is no easy way to do this, especially when datasets have different shapes</p>
<p dir="auto"><em>Original ticket <a href="http://projects.scipy.org/numpy/ticket/1107" rel="nofollow">http://projects.scipy.org/numpy/ticket/1107</a> on 2009-05-13 by trac user ctenney, assigned to unknown.</em></p>
<p dir="auto">Problem:<br>
I wanted to be able to easily read multiple blocks of array data into Numpy from a single text file (e.g. xyz positions of a collection of atoms at different times). I was unable to find an existing tool to accomplish this.</p>
<p dir="auto">Solution:<br>
I modified the loadtxt function in numpy/lib/io.py (diff pasted below).</p>
<p dir="auto">Summary:<br>
I added two new parameter options: 'numcols' and 'datablocks'. The defaults were chosen to reproduce previous behavior.</p>
<p dir="auto">'numcols' is an optional int that defaults to None. If set, 'numcols' specifies the number of columns in a valid data line. This is useful for filtering out non-comment header info (particularly from files with multiple blocks of data).</p>
<p dir="auto">'datablocks' is a boolean that defaults to False. If True, data blocks within the file are assumed to be separated by two or more non-data lines (e.g. blank, comment, or invalid 'numcols'). An additional leading dimension is added to each returned array if two or more data blocks are present. (e.g. If X = np.loadtxt(..., datablocks=True), then X[0] is the first block, X[1] the second, et cetera; and X.ravel returns the same result as np.loadtxt(..., datablocks=False).) The result is unchanged for single-block data sets.</p>
<p dir="auto">Notes:</p>
<ul dir="auto">
<li>I successfully ran the changes against numpy/lib/tests/test_io.py.</li>
<li>When faced with multi-block datasets, use of either or both of these options makes plotting in matplotlib almost as easy as plotting in gnuplot, which is particularly handy for data exploration via ipython -pylab.</li>
</ul>
<p dir="auto">diff for numpy/lib/io.py for v1.3.0:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="332,333c332
&lt;             skiprows=0, usecols=None, unpack=False,
&lt;             numcols=None, datablocks=False):

---
&gt;             skiprows=0, usecols=None, unpack=False):
368,379d366
&lt;     numcols : int
&lt;         If set, 'numcols' specifies the number of columns in a valid data line.
&lt;         This is useful for filtering out non-comment header info (particularly
&lt;         from files with multiple blocks of data).
&lt;     datablocks : bool
&lt;         If True, data blocks within the file are assumed to be separated by two
&lt;         or more non-data lines (e.g. blank, comment, or invalid 'numcols'). An
&lt;         additional leading dimension is added to each returned array if two or
&lt;         more data blocks are present. (e.g. If X = np.loadtxt(...,
&lt;         datablocks=True), then X[0] is the first block, X[1] the second, et
&lt;         cetera; and X.ravel returns the same result as np.loadtxt(...,
&lt;         datablocks=False).) The result is unchanged for single-block data sets.
471,472d457
&lt;             if numcols and len(first_vals) != numcols:
&lt;                 first_vals = None
495,497d479
&lt;         X.append([])
&lt;         block = 0
&lt;         blanks = 0
501d482
&lt;                 blanks += 1
503,509d483
&lt;             elif numcols and len(vals) != numcols:
&lt;                 blanks += 1
&lt;                 continue
&lt;             elif datablocks and blanks &gt;= 2:
&lt;                 block += 1
&lt;                 X.append([])
&lt;             blanks = 0
515,516c489
&lt;             X[block].append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
&lt; 

---
&gt;             X.append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
536c509
&lt;         return np.rollaxis(X, -1, 0)

---
&gt;         return X.T"><pre class="notranslate"><code class="notranslate">332,333c332
&lt;             skiprows=0, usecols=None, unpack=False,
&lt;             numcols=None, datablocks=False):

---
&gt;             skiprows=0, usecols=None, unpack=False):
368,379d366
&lt;     numcols : int
&lt;         If set, 'numcols' specifies the number of columns in a valid data line.
&lt;         This is useful for filtering out non-comment header info (particularly
&lt;         from files with multiple blocks of data).
&lt;     datablocks : bool
&lt;         If True, data blocks within the file are assumed to be separated by two
&lt;         or more non-data lines (e.g. blank, comment, or invalid 'numcols'). An
&lt;         additional leading dimension is added to each returned array if two or
&lt;         more data blocks are present. (e.g. If X = np.loadtxt(...,
&lt;         datablocks=True), then X[0] is the first block, X[1] the second, et
&lt;         cetera; and X.ravel returns the same result as np.loadtxt(...,
&lt;         datablocks=False).) The result is unchanged for single-block data sets.
471,472d457
&lt;             if numcols and len(first_vals) != numcols:
&lt;                 first_vals = None
495,497d479
&lt;         X.append([])
&lt;         block = 0
&lt;         blanks = 0
501d482
&lt;                 blanks += 1
503,509d483
&lt;             elif numcols and len(vals) != numcols:
&lt;                 blanks += 1
&lt;                 continue
&lt;             elif datablocks and blanks &gt;= 2:
&lt;                 block += 1
&lt;                 X.append([])
&lt;             blanks = 0
515,516c489
&lt;             X[block].append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
&lt; 

---
&gt;             X.append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
536c509
&lt;         return np.rollaxis(X, -1, 0)

---
&gt;         return X.T
</code></pre></div>