files: Header section metadata
  =======================
  
  Tutorial
  --------
  
  One of the primary motivations in writing lasio was to be able to reliably
  parse LAS header sections. This is working well for LAS 1.2 and 2.0
  files, and partially for LAS 3.0 files. 
  
  .. note::
  
     lasio does not require LAS files to be strictly compliant with the standards,
     and you should not expect lasio to raise an exception or error for files which
     are clearly not conforming to the standards. The goal of lasio is to parse
     metadata and data quietly, not to fail unnecessarily.
  
  .. code-block:: python
  
      >>> import lasio.examples
      >>> las = lasio.examples.open('6038187_v1.2_short.las')
  
  The header sections are stored in the dictionary ``las.sections``:
  
  .. code-block:: python
  
      >>> type(las.sections)
      dict
      >>> las.sections.keys()
      dict_keys(['Version', 'Well', 'Curves', 'Parameter', 'Other'])
  
  These are special names reserved for LAS 1.2 and 2.0 files, as defined by the
  standard. Non-standard header sections are also allowed but not fully parsed.
  
  ==============================  =======================================  ===================================================================
  LAS file                        Read in as                               References in ``LASFile``
  ==============================  =======================================  ===================================================================
  ``~v`` or ``~V``                :class:`lasio.SectionItems`              ``LASFile.version`` and ``LASFile.sections['Version']``
  ``~w`` or ``~W``                :class:`lasio.SectionItems`              ``LASFile.well`` and ``LASFile.sections['Well']``
  ``~c`` or ``~C``                :class:`lasio.SectionItems`              ``LASFile.curves`` and ``LASFile.sections['Curves']``
  ``~p`` or ``~P``                :class:`lasio.SectionItems`              ``LASFile.params`` and ``LASFile.sections['Parameter']``
  ``~o`` or ``~O``                ``str``                                  ``LASFile.other`` and ``LASFile.sections['Other']``
  ``~extra section``              ``str``                                  ``LASFile.sections['extra section']``
  ``~a`` or ``~A``                :class:`numpy.ndarray`                   ``LASFile.data`` or each column is in ``LASFile.curves[...].data``
  ==============================  =======================================  ===================================================================
  
  For example:
  
  .. code-block:: python
  
      >>> las.sections['Version']
      [HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA"),
       HeaderItem(mnemonic="WRAP", unit="", value="NO", descr="ONE LINE PER DEPTH STE")]
  
      >>> las.version
      [HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA"),
       HeaderItem(mnemonic="WRAP", unit="", value="NO", descr="ONE LINE PER DEPTH STE")]
  
  Sections themselves are represented by :class:`lasio.SectionItems`
  objects. This is a ``list`` which has been extended to allow you to access the
  items within by their mnemonic:
  
  .. code-block:: python
  
      >>> las.version.VERS
      HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA")
      >>> las.version['VERS']
      HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA")
      >>> las.version[0]
      HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA")
  
  As you can see, either attribute-style or item-style access is fine.
  
  Let's take a look at the next special section, ``~W``:
  
  .. code-block:: python
  
      >>> las.well
       [HeaderItem(mnemonic="STRT", unit="M", value="0.05", descr="FIRST INDEX VALUE"),
        HeaderItem(mnemonic="STOP", unit="M", value="136.6", descr="LAST INDEX VALUE"),
        HeaderItem(mnemonic="STEP", unit="M", value="0.05", descr="STEP"),
        HeaderItem(mnemonic="NULL", unit="", value="-99999", descr="NULL VALUE"),
        HeaderItem(mnemonic="COMP", unit="", value="", descr="COMP"),
        HeaderItem(mnemonic="WELL", unit="", value="Scorpio E1", descr="WELL"),
        HeaderItem(mnemonic="FLD", unit="", value="", descr=""),
        HeaderItem(mnemonic="LOC", unit="", value="Mt Eba", descr="LOC"),
        HeaderItem(mnemonic="SRVC", unit="", value="", descr=""),
        HeaderItem(mnemonic="CTRY", unit="", value="", descr=""),
        HeaderItem(mnemonic="STAT", unit="", value="SA", descr="STAT"),
        HeaderItem(mnemonic="CNTY", unit="", value="", descr=""),
        HeaderItem(mnemonic="DATE", unit="", value="15/03/2015", descr="DATE"),
        HeaderItem(mnemonic="UWI", unit="", value="6038-187", descr="WUNT")]
  
  The CTRY item is blank. We will set it:
  
  .. code-block:: python
  
      >>> las.well.CTRY = 'Australia'
      >>> las.well.CTRY
      HeaderItem(mnemonic="CTRY", unit="", value="Australia", descr="")
  
  Notice that :class:`lasio.SectionItems` plays a little trick here. It actually
  sets the ``header_item.value`` attribute, instead of replacing the entire
  :class:`lasio.HeaderItem` object.
  
  You can set any of the attributes directly. Let's take an example from the ``~C`` section:
  
  .. code-block:: python
  
      >>> las.curves
      [CurveItem(mnemonic="DEPT", unit="M", value="", descr="DEPTH", original_mnemonic="DEPT", data.shape=(121,)),
       CurveItem(mnemonic="CALI", unit="MM", value="", descr="CALI", original_mnemonic="CALI", data.shape=(121,)),
       CurveItem(mnemonic="DFAR", unit="G/CM3", value="", descr="DFAR", original_mnemonic="DFAR", data.shape=(121,)),
       CurveItem(mnemonic="DNEAR", unit="G/CM3", value="", descr="DNEAR", original_mnemonic="DNEAR", data.shape=(121,)),
       CurveItem(mnemonic="GAMN", unit="GAPI", value="", descr="GAMN", original_mnemonic="GAMN", data.shape=(121,)),
       CurveItem(mnemonic="NEUT", unit="CPS", value="", descr="NEUT", original_mnemonic="NEUT", data.shape=(121,)),
       CurveItem(mnemonic="PR", unit="OHM/M", value="", descr="PR", original_mnemonic="PR", data.shape=(121,)),
       CurveItem(mnemonic="SP", unit="MV", value="", descr="SP", original_mnemonic="SP", data.shape=(121,)),
       CurveItem(mnemonic="COND", unit="MS/M", value="", descr="COND", original_mnemonic="COND", data.shape=(121,))]
      >>> las.curves.PR.unit = 'ohmm'
      >>> las.curves.PR
      CurveItem(mnemonic="PR", unit="ohmm", value="", descr="PR", original_mnemonic="PR", data.shape=(121,))
  
  Now let's look more closely at how to manipulate and add or remove items from
  a section.
  
  .. code-block:: python
  
      In [195]: las.params
      Out[195]:
      [HeaderItem(mnemonic="BS", unit="", value="216 mm", descr="BS"),
       HeaderItem(mnemonic="JOBN", unit="", value="", descr="JOBN"),
       HeaderItem(mnemonic="WPMT", unit="", value="", descr="WPMT"),
       HeaderItem(mnemonic="AGL", unit="", value="", descr="AGL"),
       HeaderItem(mnemonic="PURP", unit="", value="Cased hole stratigraphy", descr="P"),
       HeaderItem(mnemonic="X", unit="", value="560160", descr="X"),
       HeaderItem(mnemonic="CSGL", unit="", value="0 m - 135 m", descr="CSGL"),
       HeaderItem(mnemonic="UNIT", unit="", value="", descr="UNIT"),
       HeaderItem(mnemonic="Y", unit="", value="6686430", descr="Y"),
       HeaderItem(mnemonic="TDL", unit="", value="135.2 m", descr="TDL"),
       HeaderItem(mnemonic="PROD", unit="", value="", descr="PROD"),
       HeaderItem(mnemonic="MUD", unit="", value="Water", descr="MUD"),
       HeaderItem(mnemonic="CSGS", unit="", value="100 mm", descr="CSGS"),
       HeaderItem(mnemonic="ENG", unit="", value="", descr="ENG"),
       HeaderItem(mnemonic="STEP", unit="", value="5 cm", descr="STEP"),
       HeaderItem(mnemonic="FLUIDLEVEL", unit="", value="54 m", descr="FluidLevel"),
       HeaderItem(mnemonic="CSGT", unit="", value="PVC", descr="CSGT"),
       HeaderItem(mnemonic="WIT", unit="", value="", descr="WIT"),
       HeaderItem(mnemonic="EREF", unit="", value="", descr="EREF"),
       HeaderItem(mnemonic="PROJ", unit="", value="", descr="PROJ"),
       HeaderItem(mnemonic="ZONE", unit="", value="53J", descr="ZONE"),
       HeaderItem(mnemonic="DREF", unit="", value="GL", descr="DREF"),
       HeaderItem(mnemonic="TDD", unit="", value="136 m", descr="TDD")]
  
  We want to rename the DREF mnemonic as LMF. We can do so by changing the
  ``header_item.mnemonic`` attribute.
  
  .. code-block:: python
  
      >>> las.params.DREF.mnemonic = 'LMF'
      >>> las.params
      [HeaderItem(mnemonic="BS", unit="", value="216 mm", descr="BS"),
       HeaderItem(mnemonic="JOBN", unit="", value="", descr="JOBN"),
       HeaderItem(mnemonic="WPMT", unit="", value="", descr="WPMT"),
       HeaderItem(mnemonic="AGL", unit="", value="", descr="AGL"),
       HeaderItem(mnemonic="PURP", unit="", value="Cased hole stratigraphy", descr="P"),
       HeaderItem(mnemonic="X", unit="", value="560160", descr="X"),
       HeaderItem(mnemonic="CSGL", unit="", value="0 m - 135 m", descr="CSGL"),
       HeaderItem(mnemonic="UNIT", unit="", value="", descr="UNIT"),
       HeaderItem(mnemonic="Y", unit="", value="6686430", descr="Y"),
       HeaderItem(mnemonic="TDL", unit="", value="135.2 m", descr="TDL"),
       HeaderItem(mnemonic="PROD", unit="", value="", descr="PROD"),
       HeaderItem(mnemonic="MUD", unit="", value="Water", descr="MUD"),
       HeaderItem(mnemonic="CSGS", unit="", value="100 mm", descr="CSGS"),
       HeaderItem(mnemonic="ENG", unit="", value="", descr="ENG"),
       HeaderItem(mnemonic="STEP", unit="", value="5 cm", descr="STEP"),
       HeaderItem(mnemonic="FLUIDLEVEL", unit="", value="54 m", descr="FluidLevel"),
       HeaderItem(mnemonic="CSGT", unit="", value="PVC", descr="CSGT"),
       HeaderItem(mnemonic="WIT", unit="", value="", descr="WIT"),
       HeaderItem(mnemonic="EREF", unit="", value="", descr="EREF"),
       HeaderItem(mnemonic="PROJ", unit="", value="", descr="PROJ"),
       HeaderItem(mnemonic="ZONE", unit="", value="53J", descr="ZONE"),
       HeaderItem(mnemonic="LMF", unit="", value="GL", descr="DREF"),
       HeaderItem(mnemonic="TDD", unit="", value="136 m", descr="TDD")]
  
  And now we need to add a new mnemonic.
  
  .. code-block:: python
  
      >>> las.params.DRILL = lasio.HeaderItem(mnemonic='DRILL', value='John Smith', descr='Driller on site')
      >>> las.params
      [HeaderItem(mnemonic="BS", unit="", value="216 mm", descr="BS"),
       HeaderItem(mnemonic="JOBN", unit="", value="", descr="JOBN"),
       HeaderItem(mnemonic="WPMT", unit="", value="", descr="WPMT"),
       HeaderItem(mnemonic="AGL", unit="", value="", descr="AGL"),
       HeaderItem(mnemonic="PURP", unit="", value="Cased hole stratigraphy", descr="P"),
       HeaderItem(mnemonic="X", unit="", value="560160", descr="X"),
       HeaderItem(mnemonic="CSGL", unit="", value="0 m - 135 m", descr="CSGL"),
       HeaderItem(mnemonic="UNIT", unit="", value="", descr="UNIT"),
       HeaderItem(mnemonic="Y", unit="", value="6686430", descr="Y"),
       HeaderItem(mnemonic="TDL", unit="", value="135.2 m", descr="TDL"),
       HeaderItem(mnemonic="PROD", unit="", value="", descr="PROD"),
       HeaderItem(mnemonic="MUD", unit="", value="Water", descr="MUD"),
       HeaderItem(mnemonic="CSGS", unit="", value="100 mm", descr="CSGS"),
       HeaderItem(mnemonic="ENG", unit="", value="", descr="ENG"),
       HeaderItem(mnemonic="STEP", unit="", value="5 cm", descr="STEP"),
       HeaderItem(mnemonic="FLUIDLEVEL", unit="", value="54 m", descr="FluidLevel"),
       HeaderItem(mnemonic="CSGT", unit="", value="PVC", descr="CSGT"),
       HeaderItem(mnemonic="WIT", unit="", value="", descr="WIT"),
       HeaderItem(mnemonic="EREF", unit="", value="", descr="EREF"),
       HeaderItem(mnemonic="PROJ", unit="", value="", descr="PROJ"),
       HeaderItem(mnemonic="ZONE", unit="", value="53J", descr="ZONE"),
       HeaderItem(mnemonic="LMF", unit="", value="GL", descr="DREF"),
       HeaderItem(mnemonic="TDD", unit="", value="136 m", descr="TDD"),
       HeaderItem(mnemonic="DRILL", unit="", value="John Smith", descr="Driller on si")]
  
  Bingo.
  
  What if we want to delete or remove an item? You can delete items the same way you
  would remove an item from a dictionary. Let's remove the item we just added (DRILL):
  
  .. code-block:: python
  
      >>> del las.well["DRILL"]
      
  There are methods intended for removing curves. Say you want to remove the PR curve:
  
  .. code-block:: python
  
      >>> las.delete_curve("PR")
      [CurveItem(mnemonic="DEPT", unit="M", value="", descr="DEPTH", original_mnemonic="DEPT", data.shape=(121,)),
       CurveItem(mnemonic="CALI", unit="MM", value="", descr="CALI", original_mnemonic="CALI", data.shape=(121,)),
       CurveItem(mnemonic="DFAR", unit="G/CM3", value="", descr="DFAR", original_mnemonic="DFAR", data.shape=(121,)),
       CurveItem(mnemonic="DNEAR", unit="G/CM3", value="", descr="DNEAR", original_mnemonic="DNEAR", data.shape=(121,)),
       CurveItem(mnemonic="GAMN", unit="GAPI", value="", descr="GAMN", original_mnemonic="GAMN", data.shape=(121,)),
       CurveItem(mnemonic="NEUT", unit="CPS", value="", descr="NEUT", original_mnemonic="NEUT", data.shape=(121,)),
       CurveItem(mnemonic="SP", unit="MV", value="", descr="SP", original_mnemonic="SP", data.shape=(121,)),
       CurveItem(mnemonic="COND", unit="MS/M", value="", descr="COND", original_mnemonic="COND", data.shape=(121,))]
  
  
  .. warning:: 
  
      Common mistake!
  
      A common job is to iterate through the curves and remove all but a few that you are
      interested in. When doing this, be careful to iterate over a **copy** of the curves
      section. See example below.
  
  .. code-block:: python
  
      >>> keep_curves = ['DEPT', 'DFAR', 'DNEAR']
      >>> for curve in las.curves[:]:
      ...     if curve.mnemonic not in keep_curves:
      ...        las.delete_curve(curve.mnemonic)
      ... 
      >>> las.curves
      [CurveItem(mnemonic="DEPT", unit="M", value="", descr="DEPTH", original_mnemonic="DEPT", data.shape=(121,)),
       CurveItem(mnemonic="DFAR", unit="G/CM3", value="", descr="DFAR", original_mnemonic="DFAR", data.shape=(121,)),
       CurveItem(mnemonic="DNEAR", unit="G/CM3", value="", descr="DNEAR", original_mnemonic="DNEAR", data.shape=(121,))]
  
  Handling special cases of header lines
  --------------------------------------
  
  lasio will do its best to read every line from the header section. Some examples
  follow for unusual formattings:
  
  Comment lines mixed with header lines
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  lasio will, by default, treat header lines starting with a "#" hash string as a
  comment line and ignore it. Spaces before the "#" are stripped off before
  checking for the "#".
  
  To modify which strings indicate comment lines to ignore pass an
  ignore_comments tuple to ``lasio.read()`` or ``lasio.examples.open()``.
  
  Example:
    ``lasio.read(file, ignore_comments=("#", "%MyComment")``
  
  Lines without periods
  ~~~~~~~~~~~~~~~~~~~~~
  
  For example take these lines from a LAS file header section:
  
  .. code-block:: none
  
                DRILLED  :12/11/2010
                PERM DAT :1
                TIME     :14:00:32
                HOLE DIA :85.7
  
  These lines are missing periods between the mnemonic and colon, e.g. a properly
  formatted version would be ``DRILLED. :12/11/2010``.
  
  However, lasio will parse them silently, and correctly, e.g. for the last
  line the mnemonic will be ``HOLE DIA`` and the value will be ``85.7``, with the
  description blank.
  
  Lines with colons in the mnemonic and description
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Colons are used as a delimiter, but colons can also occur inside the unit, value, and
  description fields in a LAS file header. Take this line as an example:
  
  .. code-block::
  
      TIML.hh:mm 23:15 23-JAN-2001:   Time Logger: At Bottom
  
  lasio will parse this correctly such that the unit is ``hh:mm``, the value is
  ``23:15 21-JAN-2001``, and the description is ``Time Logger: At Bottom``.
  
  Units containing periods
  ~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Similarly, periods are used as delimiters, but can also occur as part of the
  unit field's value, such as in the case of a unit of tenths of an inch (``.1IN``):
  
  .. code-block:: none
  
      TDEP  ..1IN                      :  0.1-in
  
  lasio will parse the mnemonic as ``TDEP`` and the unit as ``.1IN``.
  
  If there are two adjoining periods, the same behaviour applies:
  
  .. code-block:: none
  
      TDEP..1IN                      :  0.1-in
  
  lasio parses this line as having mnemonic ``TDEP`` and unit ``.1IN``.
  
  Special case for units which contain spaces
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Normally, any whitespace following the unit in a LAS header line delimits
  the unit from the value. lasio has a special exception for units which may
  appear with a space. Currently the only one recognised is ``1000 lbf``:
  
  .. code-block:: none
  
      HKLA            .1000 lbf                                  :(RT)
  
  This is parsed as mnemonic ``HKLA``, unit ``1000 lbf``, and value blank, contrary
  to the usual behaviour which would result in unit ``1000`` and value ``lbf``.
  
  Please raise a `GitHub issue <https://github.com/kinverarity1/lasio/issues/new/choose>`__ 
  for any other units which should be handled in this way.
  
  Mnemonics which contain a period
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  As with other LAS file parsers, lasio does not parse mnemonics which contain
  a period - instead, anything after the period will be parsed as the unit:
  
  .. code-block::
  
      SP.COND .US/M                      :  EC at 25 deg C
  
  results in mnemonic ``SP``, unit ``COND``, and value ``.US/CM``.
  
  .. warning::
  
      These files are non-conforming, and difficult to anticipate.
  
  Handling errors silently (``ignore_header_errors=True``)
  --------------------------------------------------------
  
  Sometimes lasio cannot make sense of a header line at all. For example:
  
  .. code-block:: none
  
      API       .                                          : API Number     (required if CTRY = US)
      "# Surface Coords: 1,000' FNL & 2,000' FWL"
      LATI      .DEG                                       : Latitude  - see Surface Coords comment above
      LONG      .DEG                                       : Longitude - see Surface Coords comment above
  
  The line with ``"`` causes an exception to be raised by default. 
  
  Another example is this ~Param section in a LAS file:
  
  .. code-block:: none
  
     ~PARAMETER INFORMATION
     DEPTH     DT       RHOB     NPHI     SFLU     SFLA      ILM      ILD
  
  This isn't a header line, and cannot be parsed as such. It results in a
  ``LASHeaderError`` exception being raised:
  
  .. code-block:: python
  
      >>> las = lasio.examples.open('dodgy_param_sect.las', ignore_header_errors=False)
      Unable to parse line as LAS header: DEPTH     DT       RHOB     NPHI     SFLU     SFLA      ILM      ILD
      Traceback (most
  
  [... the rest is omitted ...]
files: docs/source/header-section.rst
files: Header section metadata
  =======================
  
  Tutorial
  --------
  
  One of the primary motivations in writing lasio was to be able to reliably
  parse LAS header sections. This is working well for LAS 1.2 and 2.0
  files, and partially for LAS 3.0 files. 
  
  .. note::
  
     lasio does not require LAS files to be strictly compliant with the standards,
     and you should not expect lasio to raise an exception or error for files which
     are clearly not conforming to the standards. The goal of lasio is to parse
     metadata and data quietly, not to fail unnecessarily.
  
  .. code-block:: python
  
      >>> import lasio.examples
      >>> las = lasio.examples.open('6038187_v1.2_short.las')
  
  The header sections are stored in the dictionary ``las.sections``:
  
  .. code-block:: python
  
      >>> type(las.sections)
      dict
      >>> las.sections.keys()
      dict_keys(['Version', 'Well', 'Curves', 'Parameter', 'Other'])
  
  These are special names reserved for LAS 1.2 and 2.0 files, as defined by the
  standard. Non-standard header sections are also allowed but not fully parsed.
  
  ==============================  =======================================  ===================================================================
  LAS file                        Read in as                               References in ``LASFile``
  ==============================  =======================================  ===================================================================
  ``~v`` or ``~V``                :class:`lasio.SectionItems`              ``LASFile.version`` and ``LASFile.sections['Version']``
  ``~w`` or ``~W``                :class:`lasio.SectionItems`              ``LASFile.well`` and ``LASFile.sections['Well']``
  ``~c`` or ``~C``                :class:`lasio.SectionItems`              ``LASFile.curves`` and ``LASFile.sections['Curves']``
  ``~p`` or ``~P``                :class:`lasio.SectionItems`              ``LASFile.params`` and ``LASFile.sections['Parameter']``
  ``~o`` or ``~O``                ``str``                                  ``LASFile.other`` and ``LASFile.sections['Other']``
  ``~extra section``              ``str``                                  ``LASFile.sections['extra section']``
  ``~a`` or ``~A``                :class:`numpy.ndarray`                   ``LASFile.data`` or each column is in ``LASFile.curves[...].data``
  ==============================  =======================================  ===================================================================
  
  For example:
  
  .. code-block:: python
  
      >>> las.sections['Version']
      [HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA"),
       HeaderItem(mnemonic="WRAP", unit="", value="NO", descr="ONE LINE PER DEPTH STE")]
  
      >>> las.version
      [HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA"),
       HeaderItem(mnemonic="WRAP", unit="", value="NO", descr="ONE LINE PER DEPTH STE")]
  
  Sections themselves are represented by :class:`lasio.SectionItems`
  objects. This is a ``list`` which has been extended to allow you to access the
  items within by their mnemonic:
  
  .. code-block:: python
  
      >>> las.version.VERS
      HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA")
      >>> las.version['VERS']
      HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA")
      >>> las.version[0]
      HeaderItem(mnemonic="VERS", unit="", value="2.0", descr="CWLS LOG ASCII STANDA")
  
  As you can see, either attribute-style or item-style access is fine.
  
  Let's take a look at the next special section, ``~W``:
  
  .. code-block:: python
  
      >>> las.well
       [HeaderItem(mnemonic="STRT", unit="M", value="0.05", descr="FIRST INDEX VALUE"),
        HeaderItem(mnemonic="STOP", unit="M", value="136.6", descr="LAST INDEX VALUE"),
        HeaderItem(mnemonic="STEP", unit="M", value="0.05", descr="STEP"),
        HeaderItem(mnemonic="NULL", unit="", value="-99999", descr="NULL VALUE"),
        HeaderItem(mnemonic="COMP", unit="", value="", descr="COMP"),
        HeaderItem(mnemonic="WELL", unit="", value="Scorpio E1", descr="WELL"),
        HeaderItem(mnemonic="FLD", unit="", value="", descr=""),
        HeaderItem(mnemonic="LOC", unit="", value="Mt Eba", descr="LOC"),
        HeaderItem(mnemonic="SRVC", unit="", value="", descr=""),
        HeaderItem(mnemonic="CTRY", unit="", value="", descr=""),
        HeaderItem(mnemonic="STAT", unit="", value="SA", descr="STAT"),
        HeaderItem(mnemonic="CNTY", unit="", value="", descr=""),
        HeaderItem(mnemonic="DATE", unit="", value="15/03/2015", descr="DATE"),
        HeaderItem(mnemonic="UWI", unit="", value="6038-187", descr="WUNT")]
  
  The CTRY item is blank. We will set it:
  
  .. code-block:: python
  
      >>> las.well.CTRY = 'Australia'
      >>> las.well.CTRY
      HeaderItem(mnemonic="CTRY", unit="", value="Australia", descr="")
  
  Notice that :class:`lasio.SectionItems` plays a little trick here. It actually
  sets the ``header_item.value`` attribute, instead of replacing the entire
  :class:`lasio.HeaderItem` object.
  
  You can set any of the attributes directly. Let's take an example from the ``~C`` section:
  
  .. code-block:: python
  
      >>> las.curves
      [CurveItem(mnemonic="DEPT", unit="M", value="", descr="DEPTH", original_mnemonic="DEPT", data.shape=(121,)),
       CurveItem(mnemonic="CALI", unit="MM", value="", descr="CALI", original_mnemonic="CALI", data.shape=(121,)),
       CurveItem(mnemonic="DFAR", unit="G/CM3", value="", descr="DFAR", original_mnemonic="DFAR", data.shape=(121,)),
       CurveItem(mnemonic="DNEAR", unit="G/CM3", value="", descr="DNEAR", original_mnemonic="DNEAR", data.shape=(121,)),
       CurveItem(mnemonic="GAMN", unit="GAPI", value="", descr="GAMN", original_mnemonic="GAMN", data.shape=(121,)),
       CurveItem(mnemonic="NEUT", unit="CPS", value="", descr="NEUT", original_mnemonic="NEUT", data.shape=(121,)),
       CurveItem(mnemonic="PR", unit="OHM/M", value="", descr="PR", original_mnemonic="PR", data.shape=(121,)),
       CurveItem(mnemonic="SP", unit="MV", value="", descr="SP", original_mnemonic="SP", data.shape=(121,)),
       CurveItem(mnemonic="COND", unit="MS/M", value="", descr="COND", original_mnemonic="COND", data.shape=(121,))]
      >>> las.curves.PR.unit = 'ohmm'
      >>> las.curves.PR
      CurveItem(mnemonic="PR", unit="ohmm", value="", descr="PR", original_mnemonic="PR", data.shape=(121,))
  
  Now let's look more closely at how to manipulate and add or remove items from
  a section.
  
  .. code-block:: python
  
      In [195]: las.params
      Out[195]:
      [HeaderItem(mnemonic="BS", unit="", value="216 mm", descr="BS"),
       HeaderItem(mnemonic="JOBN", unit="", value="", descr="JOBN"),
       HeaderItem(mnemonic="WPMT", unit="", value="", descr="WPMT"),
       HeaderItem(mnemonic="AGL", unit="", value="", descr="AGL"),
       HeaderItem(mnemonic="PURP", unit="", value="Cased hole stratigraphy", descr="P"),
       HeaderItem(mnemonic="X", unit="", value="560160", descr="X"),
       HeaderItem(mnemonic="CSGL", unit="", value="0 m - 135 m", descr="CSGL"),
       HeaderItem(mnemonic="UNIT", unit="", value="", descr="UNIT"),
       HeaderItem(mnemonic="Y", unit="", value="6686430", descr="Y"),
       HeaderItem(mnemonic="TDL", unit="", value="135.2 m", descr="TDL"),
       HeaderItem(mnemonic="PROD", unit="", value="", descr="PROD"),
       HeaderItem(mnemonic="MUD", unit="", value="Water", descr="MUD"),
       HeaderItem(mnemonic="CSGS", unit="", value="100 mm", descr="CSGS"),
       HeaderItem(mnemonic="ENG", unit="", value="", descr="ENG"),
       HeaderItem(mnemonic="STEP", unit="", value="5 cm", descr="STEP"),
       HeaderItem(mnemonic="FLUIDLEVEL", unit="", value="54 m", descr="FluidLevel"),
       HeaderItem(mnemonic="CSGT", unit="", value="PVC", descr="CSGT"),
       HeaderItem(mnemonic="WIT", unit="", value="", descr="WIT"),
       HeaderItem(mnemonic="EREF", unit="", value="", descr="EREF"),
       HeaderItem(mnemonic="PROJ", unit="", value="", descr="PROJ"),
       HeaderItem(mnemonic="ZONE", unit="", value="53J", descr="ZONE"),
       HeaderItem(mnemonic="DREF", unit="", value="GL", descr="DREF"),
       HeaderItem(mnemonic="TDD", unit="", value="136 m", descr="TDD")]
  
  We want to rename the DREF mnemonic as LMF. We can do so by changing the
  ``header_item.mnemonic`` attribute.
  
  .. code-block:: python
  
      >>> las.params.DREF.mnemonic = 'LMF'
      >>> las.params
      [HeaderItem(mnemonic="BS", unit="", value="216 mm", descr="BS"),
       HeaderItem(mnemonic="JOBN", unit="", value="", descr="JOBN"),
       HeaderItem(mnemonic="WPMT", unit="", value="", descr="WPMT"),
       HeaderItem(mnemonic="AGL", unit="", value="", descr="AGL"),
       HeaderItem(mnemonic="PURP", unit="", value="Cased hole stratigraphy", descr="P"),
       HeaderItem(mnemonic="X", unit="", value="560160", descr="X"),
       HeaderItem(mnemonic="CSGL", unit="", value="0 m - 135 m", descr="CSGL"),
       HeaderItem(mnemonic="UNIT", unit="", value="", descr="UNIT"),
       HeaderItem(mnemonic="Y", unit="", value="6686430", descr="Y"),
       HeaderItem(mnemonic="TDL", unit="", value="135.2 m", descr="TDL"),
       HeaderItem(mnemonic="PROD", unit="", value="", descr="PROD"),
       HeaderItem(mnemonic="MUD", unit="", value="Water", descr="MUD"),
       HeaderItem(mnemonic="CSGS", unit="", value="100 mm", descr="CSGS"),
       HeaderItem(mnemonic="ENG", unit="", value="", descr="ENG"),
       HeaderItem(mnemonic="STEP", unit="", value="5 cm", descr="STEP"),
       HeaderItem(mnemonic="FLUIDLEVEL", unit="", value="54 m", descr="FluidLevel"),
       HeaderItem(mnemonic="CSGT", unit="", value="PVC", descr="CSGT"),
       HeaderItem(mnemonic="WIT", unit="", value="", descr="WIT"),
       HeaderItem(mnemonic="EREF", unit="", value="", descr="EREF"),
       HeaderItem(mnemonic="PROJ", unit="", value="", descr="PROJ"),
       HeaderItem(mnemonic="ZONE", unit="", value="53J", descr="ZONE"),
       HeaderItem(mnemonic="LMF", unit="", value="GL", descr="DREF"),
       HeaderItem(mnemonic="TDD", unit="", value="136 m", descr="TDD")]
  
  And now we need to add a new mnemonic.
  
  .. code-block:: python
  
      >>> las.params.DRILL = lasio.HeaderItem(mnemonic='DRILL', value='John Smith', descr='Driller on site')
      >>> las.params
      [HeaderItem(mnemonic="BS", unit="", value="216 mm", descr="BS"),
       HeaderItem(mnemonic="JOBN", unit="", value="", descr="JOBN"),
       HeaderItem(mnemonic="WPMT", unit="", value="", descr="WPMT"),
       HeaderItem(mnemonic="AGL", unit="", value="", descr="AGL"),
       HeaderItem(mnemonic="PURP", unit="", value="Cased hole stratigraphy", descr="P"),
       HeaderItem(mnemonic="X", unit="", value="560160", descr="X"),
       HeaderItem(mnemonic="CSGL", unit="", value="0 m - 135 m", descr="CSGL"),
       HeaderItem(mnemonic="UNIT", unit="", value="", descr="UNIT"),
       HeaderItem(mnemonic="Y", unit="", value="6686430", descr="Y"),
       HeaderItem(mnemonic="TDL", unit="", value="135.2 m", descr="TDL"),
       HeaderItem(mnemonic="PROD", unit="", value="", descr="PROD"),
       HeaderItem(mnemonic="MUD", unit="", value="Water", descr="MUD"),
       HeaderItem(mnemonic="CSGS", unit="", value="100 mm", descr="CSGS"),
       HeaderItem(mnemonic="ENG", unit="", value="", descr="ENG"),
       HeaderItem(mnemonic="STEP", unit="", value="5 cm", descr="STEP"),
       HeaderItem(mnemonic="FLUIDLEVEL", unit="", value="54 m", descr="FluidLevel"),
       HeaderItem(mnemonic="CSGT", unit="", value="PVC", descr="CSGT"),
       HeaderItem(mnemonic="WIT", unit="", value="", descr="WIT"),
       HeaderItem(mnemonic="EREF", unit="", value="", descr="EREF"),
       HeaderItem(mnemonic="PROJ", unit="", value="", descr="PROJ"),
       HeaderItem(mnemonic="ZONE", unit="", value="53J", descr="ZONE"),
       HeaderItem(mnemonic="LMF", unit="", value="GL", descr="DREF"),
       HeaderItem(mnemonic="TDD", unit="", value="136 m", descr="TDD"),
       HeaderItem(mnemonic="DRILL", unit="", value="John Smith", descr="Driller on si")]
  
  Bingo.
  
  What if we want to delete or remove an item? You can delete items the same way you
  would remove an item from a dictionary. Let's remove the item we just added (DRILL):
  
  .. code-block:: python
  
      >>> del las.well["DRILL"]
      
  There are methods intended for removing curves. Say you want to remove the PR curve:
  
  .. code-block:: python
  
      >>> las.delete_curve("PR")
      [CurveItem(mnemonic="DEPT", unit="M", value="", descr="DEPTH", original_mnemonic="DEPT", data.shape=(121,)),
       CurveItem(mnemonic="CALI", unit="MM", value="", descr="CALI", original_mnemonic="CALI", data.shape=(121,)),
       CurveItem(mnemonic="DFAR", unit="G/CM3", value="", descr="DFAR", original_mnemonic="DFAR", data.shape=(121,)),
       CurveItem(mnemonic="DNEAR", unit="G/CM3", value="", descr="DNEAR", original_mnemonic="DNEAR", data.shape=(121,)),
       CurveItem(mnemonic="GAMN", unit="GAPI", value="", descr="GAMN", original_mnemonic="GAMN", data.shape=(121,)),
       CurveItem(mnemonic="NEUT", unit="CPS", value="", descr="NEUT", original_mnemonic="NEUT", data.shape=(121,)),
       CurveItem(mnemonic="SP", unit="MV", value="", descr="SP", original_mnemonic="SP", data.shape=(121,)),
       CurveItem(mnemonic="COND", unit="MS/M", value="", descr="COND", original_mnemonic="COND", data.shape=(121,))]
  
  
  .. warning:: 
  
      Common mistake!
  
      A common job is to iterate through the curves and remove all but a few that you are
      interested in. When doing this, be careful to iterate over a **copy** of the curves
      section. See example below.
  
  .. code-block:: python
  
      >>> keep_curves = ['DEPT', 'DFAR', 'DNEAR']
      >>> for curve in las.curves[:]:
      ...     if curve.mnemonic not in keep_curves:
      ...        las.delete_curve(curve.mnemonic)
      ... 
      >>> las.curves
      [CurveItem(mnemonic="DEPT", unit="M", value="", descr="DEPTH", original_mnemonic="DEPT", data.shape=(121,)),
       CurveItem(mnemonic="DFAR", unit="G/CM3", value="", descr="DFAR", original_mnemonic="DFAR", data.shape=(121,)),
       CurveItem(mnemonic="DNEAR", unit="G/CM3", value="", descr="DNEAR", original_mnemonic="DNEAR", data.shape=(121,))]
  
  Handling special cases of header lines
  --------------------------------------
  
  lasio will do its best to read every line from the header section. Some examples
  follow for unusual formattings:
  
  Lines without periods
  ~~~~~~~~~~~~~~~~~~~~~
  
  For example take these lines from a LAS file header section:
  
  .. code-block:: none
  
                DRILLED  :12/11/2010
                PERM DAT :1
                TIME     :14:00:32
                HOLE DIA :85.7
  
  These lines are missing periods between the mnemonic and colon, e.g. a properly
  formatted version would be ``DRILLED. :12/11/2010``.
  
  However, lasio will parse them silently, and correctly, e.g. for the last
  line the mnemonic will be ``HOLE DIA`` and the value will be ``85.7``, with the
  description blank.
  
  Lines with colons in the mnemonic and description
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Colons are used as a delimiter, but colons can also occur inside the unit, value, and
  description fields in a LAS file header. Take this line as an example:
  
  .. code-block::
  
      TIML.hh:mm 23:15 23-JAN-2001:   Time Logger: At Bottom
  
  lasio will parse this correctly such that the unit is ``hh:mm``, the value is
  ``23:15 21-JAN-2001``, and the description is ``Time Logger: At Bottom``.
  
  Units containing periods
  ~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Similarly, periods are used as delimiters, but can also occur as part of the
  unit field's value, such as in the case of a unit of tenths of an inch (``.1IN``):
  
  .. code-block:: none
  
      TDEP  ..1IN                      :  0.1-in
  
  lasio will parse the mnemonic as ``TDEP`` and the unit as ``.1IN``.
  
  If there are two adjoining periods, the same behaviour applies:
  
  .. code-block:: none
  
      TDEP..1IN                      :  0.1-in
  
  lasio parses this line as having mnemonic ``TDEP`` and unit ``.1IN``.
  
  Special case for units which contain spaces
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  Normally, any whitespace following the unit in a LAS header line delimits
  the unit from the value. lasio has a special exception for units which may
  appear with a space. Currently the only one recognised is ``1000 lbf``:
  
  .. code-block:: none
  
      HKLA            .1000 lbf                                  :(RT)
  
  This is parsed as mnemonic ``HKLA``, unit ``1000 lbf``, and value blank, contrary
  to the usual behaviour which would result in unit ``1000`` and value ``lbf``.
  
  Please raise a `GitHub issue <https://github.com/kinverarity1/lasio/issues/new/choose>`__ 
  for any other units which should be handled in this way.
  
  Mnemonics which contain a period
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  As with other LAS file parsers, lasio does not parse mnemonics which contain
  a period - instead, anything after the period will be parsed as the unit:
  
  .. code-block::
  
      SP.COND .US/M                      :  EC at 25 deg C
  
  results in mnemonic ``SP``, unit ``COND``, and value ``.US/CM``.
  
  .. warning::
  
      These files are non-conforming, and difficult to anticipate.
  
  Handling errors silently (``ignore_header_errors=True``)
  --------------------------------------------------------
  
  Sometimes lasio cannot make sense of a header line at all. For example:
  
  .. code-block:: none
  
      API       .                                          : API Number     (required if CTRY = US)
      "# Surface Coords: 1,000' FNL & 2,000' FWL"
      LATI      .DEG                                       : Latitude  - see Surface Coords comment above
      LONG      .DEG                                       : Longitude - see Surface Coords comment above
  
  The line with ``"`` causes an exception to be raised by default. 
  
  Another example is this ~Param section in a LAS file:
  
  .. code-block:: none
  
     ~PARAMETER INFORMATION
     DEPTH     DT       RHOB     NPHI     SFLU     SFLA      ILM      ILD
  
  This isn't a header line, and cannot be parsed as such. It results in a
  ``LASHeaderError`` exception being raised:
  
  .. code-block:: python
  
      >>> las = lasio.examples.open('dodgy_param_sect.las', ignore_header_errors=False)
      Unable to parse line as LAS header: DEPTH     DT       RHOB     NPHI     SFLU     SFLA      ILM      ILD
      Traceback (most recent call last):
      File "C:\Users\kinve\code\lasio\lasio\reader.py", line 525, in parse_header_section
          values = read_line(line, section_name=parser.section_name2)
      File "C:\Users\kinve\code\lasio\lasio\reader.py", line 711, in read_line
          return read_header_line(*args, **kwargs)
      File "C:\Users\kinve\code\lasio\las
  
  [... the rest is omitted ...]
files: docs/source/header-section.rst
files: from __future__ import print_function
  
  try:  # will work in Python 3
      from collections.abc import Sequence
  except ImportError:  # Support Python 2.7
      from collections import Sequence
  
  import csv
  import json
  import logging
  import re
  import sys
  import traceback
  
  # get basestring in py3
  
  try:
      unicode = unicode
  except NameError:
      # 'unicode' is undefined, must be Python 3
      unicode = str
      basestring = (str, bytes)
  else:
      # 'unicode' exists, must be Python 2
      bytes = str
      basestring = basestring
  
  # Required third-party packages available on PyPi:
  
  import numpy as np
  
  # internal lasio imports
  
  from . import exceptions
  from .las_items import HeaderItem, CurveItem, SectionItems, OrderedDict
  from . import defaults
  from . import reader
  from . import writer
  
  logger = logging.getLogger(__name__)
  
  
  class LASFile(object):
  
      """LAS file object.
  
      Keyword Arguments:
          file_ref (file-like object, str): either a filename, an open file
              object, or a string containing the contents of a file.
  
      See these routines for additional keyword arguments you can use when
      reading in a LAS file:
  
      * :func:`lasio.reader.open_with_codecs` - manage issues relate to character
        encodings
      * :meth:`lasio.LASFile.read` - control how NULL values and errors are
        handled during parsing
  
      Attributes:
          encoding (str or None): the character encoding used when reading the
              file in from disk
  
      """
  
      def __init__(self, file_ref=None, **read_kwargs):
          super(LASFile, self).__init__()
          self._text = ""
          self.index_unit = None
          self.index_initial = None
          default_items = defaults.get_default_items()
          self.sections = {
              "Version": default_items["Version"],
              "Well": default_items["Well"],
              "Curves": default_items["Curves"],
              "Parameter": default_items["Parameter"],
              "Other": str(default_items["Other"]),
          }
  
          if not (file_ref is None):
              self.read(file_ref, **read_kwargs)
  
      def read(
          self,
          file_ref,
          ignore_header_errors=False,
          ignore_comments=("#",),
          ignore_data_comments="#",
          mnemonic_case="upper",
          ignore_data=False,
          engine="numpy",
          use_normal_engine_for_wrapped=True,
          read_policy="default",
          null_policy="strict",
          index_unit=None,
          dtypes="auto",
          **kwargs
      ):
          """Read a LAS file.
  
          Arguments:
              file_ref (file-like object, str): either a filename, an open file
                  object, or a string containing the contents of a file.
  
          Keyword Arguments:
              ignore_header_errors (bool): ignore LASHeaderErrors (False by
                  default)
              ignore_comments (sequence/str): ignore lines beginning with these
                  characters e.g. ``("#", '"')`` in header sections.
              ignore_data_comments (str): ignore lines beginning with this
                  character in data sections only.
              mnemonic_case (str): 'preserve': keep the case of HeaderItem mnemonics
                                   'upper': convert all HeaderItem mnemonics to uppercase
                                   'lower': convert all HeaderItem mnemonics to lowercase
              ignore_data (bool): if True, do not read in any of the actual data,
                  just the header metadata. False by default.
              engine (str): "normal": parse data section with normal Python reader
                  (quite slow); "numpy": parse data section with `numpy.genfromtxt` (fast).
                  By default the engine is "numpy".
              use_normal_engine_for_wrapped (bool): if header metadata indicates that
                  the file is wrapped, always use the 'normal' engine. Default is True.
                  The only reason you should use False is if speed is a very high priority
                  and you had files with metadata that incorrectly indicates they are
                  wrapped.
              read_policy (): TODO
              null_policy (str or list): see
                  http://lasio.readthedocs.io/en/latest/data-section.html#handling-invalid-data-indicators-automatically
              index_unit (str): Optionally force-set the index curve's unit to "m" or "ft"
              dtypes ("auto", dict or list): specify the data types for each curve in the
                  ~ASCII data section. If "auto", each curve will be converted to floats if
                  possible and remain as str if not. If a dict you can specify only the
                  curve mnemonics you want to convert as a key. If a list, please specify
                  data types for each curve in order. Note that the conversion currently
                  only occurs via numpy.ndarray.astype() and therefore only a few simple
                  casts will work e.g. `int`, `float`, `str`.
              encoding (str): character encoding to open file_ref with, using
                  :func:`io.open` (this is handled by
                  :func:`lasio.reader.open_with_codecs`)
              encoding_errors (str): 'strict', 'replace' (default), 'ignore' - how to
                  handle errors with encodings (see
                  `this section
                  <https://docs.python.org/3/library/codecs.html#codec-base-classes>`__
                  of the standard library's :mod:`codecs` module for more information)
                  (this is handled by :func:`lasio.reader.open_with_codecs`)
              autodetect_encoding (str or bool): default True to use
                  `chardet <https://github.com/chardet/chardet>`__/`cchardet
                  <https://github.com/PyYoshi/cChardet>`__ to detect encoding.
                  Note if set to False several common encodings will be tried but
                  chardet won't be used.
                  (this is handled by :func:`lasio.reader.open_with_codecs`)
              autodetect_encoding_chars (int/None): number of chars to read from LAS
                  file for auto-detection of encoding.
                  (this is handled by :func:`lasio.reader.open_with_codecs`)
  
          """
  
          logger.debug("Reading {}...".format(str(file_ref)))
  
          # Determine which lines to ignore:
          if ignore_comments is None:
              ignore_comments = []
          if isinstance(ignore_comments, str):
              ignore_comments = [ignore_comments]
  
          logger.debug("Ignore header lines beginning with {}".format(ignore_comments))
          logger.debug("Ignore data lines beginning with {}".format(ignore_data_comments))
  
          # Attempt to read file
          file_obj = ""
          try:
              file_obj, self.encoding = reader.open_file(file_ref, **kwargs)
  
              test_lidar = file_obj.read(4)
              if test_lidar == "LASF":
                  raise IOError("This is a LASer file (i.e. LiDAR data), not a Log ASCII Standard file")
              else:
                  file_obj.seek(0)
  
              logger.debug(
                  "Fetching substitutions for read_policy {} and null policy {}".format(
                      read_policy, null_policy
                  )
              )
              provisional_version = 2.0
              provisional_wrapped = "YES"
              provisional_null = None
              provisional_delimiter = "SPACE"
  
              section_positions = reader.find_sections_in_file(file_obj)
              logger.debug("Found {} sections".format(len(section_positions)))
              if len(section_positions) == 0:
                  raise KeyError("No ~ sections found. Is this a LAS file?")
  
              data_section_indices = []
              # This is a transitional data_section_indicies till the las30 data
              # reading can handle 1.2 and 2.0 data, then it will be merged back
              # into data_section_indices
              las3_data_section_indices = []
  
              las3_section_indicators = ["_DATA", "_PARAMETER", "_DEFINITION"]
  
              for i, (k, first_line, last_line, section_title) in enumerate(
                  section_positions
              ):
                  section_type = reader.determine_section_type(section_title)
                  logger.debug(
                      "Parsing {typ} section at lines {first_line}-{last_line} ({k} bytes) {title}".format(
                          typ=section_type,
                          title=section_title,
                          first_line=first_line + 1,
                          last_line=last_line + 1,
                          k=k,
                      )
                  )
  
                  # Read traditional LAS header item section
                  if section_type == "Header items":
                      file_obj.seek(k)
                      sct_items = reader.parse_header_items_section(
                          file_obj,
                          line_nos=(first_line, last_line),
                          version=provisional_version,
                          ignore_header_errors=ignore_header_errors,
                          mnemonic_case=mnemonic_case,
                          ignore_comments=ignore_comments,
                      )
  
                      # Update provisional statuses
                      if "VERS" in sct_items:
                          provisional_version = sct_items.VERS.value
                      if "WRAP" in sct_items:
                          provisional_wrapped = sct_items.WRAP.value
                      if "NULL" in sct_items:
                          provisional_null = sct_items.NULL.value
                      if "DLM" in sct_items:
                          provisional_delimiter = sct_items.DLM.value
  
                      # las3 sections can contain _Data, _Parameter or _Definition
                      las3_section = any(
                          [
                              section_str in section_title[1:].upper()
                              for section_str in las3_section_indicators
                          ]
                      )
  
                      # TODO: Revise so that Version, Well and Parameter(Log_Parameter)
                      # are handled the properly for LAS-1.2, LAS-2.0 LAS-3.0
                      # Set "Curves" for LAS-1.2, LAS-2.0, LAS-3.0
                      if (
                          section_title[1] == "C" and "_" not in section_title
                      ) or "~Log_Definition" in section_title:
                          self.sections["Curves"] = sct_items
                      elif (
                          section_title[1] == "P" and "_" not in section_title
                      ) or "~Log_Parameter" in section_title:
                          self.sections["Parameter"] = sct_items
                      # Set any other LAS3.0  sections
                      elif provisional_version == 3.0 and las3_section:
                          self.sections[section_title[1:]] = sct_items
                      # Set regular sections
                      elif section_title[1] == "V":
                          self.sections["Version"] = sct_items
                      elif section_title[1] == "W":
                          self.sections["Well"] = sct_items
                      else:
                          self.sections[section_title[1:]] = sct_items
  
                  # Read free-text LAS header section
                  elif section_type == "Header (other)":
                      file_obj.seek(k)
                      line_no = first_line
                      contents = []
                      for line in file_obj:
                          if line.startswith("~") and line_no == last_line:
                              break
                          if line.startswith("~"):
                              continue
                          line_no += 1
                          contents.append(line.strip("\n").strip())
                          if line_no == last_line:
                              break
                      sct_contents = "\n".join(contents)
  
                      if section_title[1] == "O":
                          self.sections["Other"] = sct_contents
                      else:
                          self.sections[section_title[1:]] = sct_contents
  
                  elif section_type == "Data":
                      logger.debug("Storing reference and returning later...")
                      data_section_indices.append(i)
  
                  # Initial stub for parsing las3 data. This is probably a
                  # transitional section that will merge with 1.2/2.0 data
                  # parsing once fully functional
                  elif section_type == "Las3_Data":
                      logger.debug("Storing Las3_Data reference and returning later...")
                      las3_data_section_indices.append(i)
  
              line_splitter = reader.define_line_splitter(provisional_delimiter)
  
              if provisional_delimiter == "COMMA":
                  read_policy = "comma-delimiter"
              regexp_subs, value_null_subs, version_NULL = reader.get_substitutions(
                  read_policy, null_policy
              )
  
              if not ignore_data:
  
                  # Override the default "numpy" parser with the 'normal' parser
                  # for these conditions:
                  # - file is wrapped
                  # - null_policy is not "strict"
                  # - dtypes is not "auto". Numpy can handle specified dtypes but
                  #   the performance decays to the 'normal' performance level.
  
                  # normal engine.
                  if provisional_wrapped == "YES" or null_policy != "strict" or dtypes != "auto":
                      if engine != "normal":
                          logger.warning("Only engine='normal' can read wrapped files")
                          if use_normal_engine_for_wrapped:
                              engine = "normal"
  
                  if (
                      len(data_section_indices) == 0
                      and len(las3_data_section_indices) > 0
                  ):
                      data_section_indices = las3_data_section_indices
                  # Check for the number of columns in each data section.
                  for k, first_line, last_line, section_title in [
                      section_positions[i] for i in data_section_indices
                  ]:
                      logger.debug("Reading data section {}".format(section_title))
  
                      file_obj.seek(k)
                      n_columns = reader.inspect_data_section(
                          file_obj,
                          (first_line, last_line),
                          regexp_subs,
                          ignore_data_comments=ignore_data_comments,
                      )
  
                      # How many curves should the reader attempt to find?
                      reader_n_columns = n_columns
                      if reader_n_columns == -1:
                          reader_n_columns = len(self.curves)
  
                      file_obj.seek(k)
  
                      # Convert dtypes passed as dict into list for all columns
                      # defaulting to float for any not specified.
                      if isinstance(dtypes, dict):
                          dtypes = [dtypes.get(c.mnemonic, float) for c in self.curves]
  
                      # Notes see 2d9e43c3 and e960998f for 'try' background
  
                      # Attempt to read the data section
                      if engine == "numpy":
                          try:
                              curves_data_gen = reader.read_data_section_iterative_numpy_engine(
                                  file_obj, (first_line, last_line)
                              )
                          except KeyboardInterrupt:
                              raise
                          except:
                              try:
                                  file_obj.seek(k)
                                  curves_data_gen = (
                                      reader.read_data_section_iterative_normal_engine(
                                          file_obj,
                                          (first_line, last_line),
                                          regexp_subs,
                                          value_null_subs,
                                          ignore_data_comments=ignore_data_comments,
                                          n_columns=reader_n_columns,
                                          dtypes=dtypes,
                                          line_splitter=line_splitter,
                                      )
                                  )
                              except KeyboardInterrupt:
                                  raise
                              except:
                                  raise exceptions.LASDataError(
                                      traceback.format_exc()[:-1]
                                      + " in data section beginning line {}".format(i + 1)
                                  )
  
                      if engine == "normal":
                          try:
                              curves_data_gen = (
                                  reader.read_data_section_iterative_normal_engine(
                                      file_obj,
                                      (first_line, last_line),
                                      regexp_subs,
                                      value_null_subs,
                                      ignore_data_comments=ignore_data_comments,
                                      n_columns=reader_n_columns,
                                      dtypes=dtypes,
                                      line_splitter=line_splitter,
                                  )
                              )
                          except KeyboardInterrupt:
                              raise
                          except:
                              raise exceptions.LASDataError(
                                  traceback.format_exc()[:-1]
                                  + " in data section beginning line {}".format(i + 1)
                              )
  
                      # Assign data to curves.
                      data_assigned_to_curves = {curve_idx: False for curve_idx in range(len(self.curves))}
                      curve_idx = 0
                      curve_length = 0
                      for curve_arr in curves_data_gen:
  
                          # Do not replace nulls in the index curve.
                          if version_NULL and curve_arr.dtype == float and curve_idx != 0:
                              logger.debug(
                                  "Replacing {} with nan in {}-th curve".format(
                                      provisional_null, curve_idx
                                  )
                              )
                              curve_arr[curve_arr == provisional_null] = np.nan
  
                          logger.debug(
                              "Assigning data {} to curve #{}".format(
                                  curve_arr, curve_idx
                              )
                          )
                          if curve_length == 0:
                              curve_length = len(curve_arr)
                          if curve_idx < len(self.curves):
                              self.curves[curve_idx].data = curve_arr
                          else:
                              logger.debug("Creating new curve")
                              curve = CurveItem(mnemonic="", data=curve_arr)
                              self.curves.append(curve)
                          data_assigned_to_curves[curve_idx] = True
                          curve_idx += 1
  
                      # Assign missing data indicators for curves which have no data in the
                      # data section.
                      for curve_idx, flag in data_assigned_to_curves.items():
                          if flag is False:
                              logger.warning(
                                  "Curve #{:.0f} '{:s}' is defined in the ~C section "
                                  "but there is no data in ~A"
                                  .format(curve_idx, self.curves[curve_idx].mnemonic)
                              )
                              self.curves[curve_idx].data = np.empty(curve_length) * np.nan
  
          finally:
              if hasattr(file_obj, "close"):
                  file_obj.close()
  
              # TODO: reimplement these warnings!!
  
              ###### logger.warning("No data section (regexp='~A') found")
              ###### logger.warning("No numerical data found inside ~A section")
  
          # Understand the depth/index unit.
  
          if "m" in str(index_unit):
              index_unit = "m"
  
          if index_unit:
              self.index_unit = index_unit
          else:
              check_units_on = []
              for mnemonic in ("STRT", "STOP", "STEP"):
                  if mnemonic in self.well:
                      check_units_on.append(self.well[mnemonic])
              if len(self.curves) > 0:
                  check_units_on.append(self.curves[0])
              matches = []
              for index_unit, possibilities in defaults.DEPTH_UNITS.items():
                  for check_unit in check_units_on:
                      if any([check_unit.unit == p for p in possibilities]) or any(
                          [check_unit.unit.upper() == p for p in possibilities]
                      ):
                          matches.append(index_unit)
              matches = set(matches)
              if len(matches) == 1:
                  self.index_unit = tuple(matches)[0]
              elif len(matches) == 0:
                  self.index_unit = None
              else:
                  logger.warning("Conflicting index units found: {}".format(matches))
                  self.index_unit = None
  
          if len(self.curves) > 0:
              self.index_initial = self.index.copy()
  
      def update_start_stop_step(self, STRT=None, STOP=None, STEP=None, fmt="%.5f"):
          """Configure or change STRT, STOP, and STEP values on the LASFile object.
  
          Keyword Arguments:
              STRT, STOP, STEP (str, int, float): value to set on the relevant
                  header item in the ~Well section - can be any
                  data type.
              fmt (str): Python format string for formatting the STRT/STOP/STEP
                  value in the situation where any of those keyword arguments
                  are None
  
          If STRT/STOP/STEP are not passed to this method, they will be automatically
          calculated from the index curve.
  
          """
  
          # If we are getting STRT and STOP from the data then format them to a
          # standard precision.
          # If they are passed in with values, don't format them because we
          # assume they are at the user's expected precision.
  
          # If the 'try' fails because self.index doesn't exist or is empty
          # then use the default or parameter values for STRT, STOP, and STEP.
          try:
              if STRT is None:
                  STRT = fmt % self.index[0]
              if STOP is None:
                  STOP = fmt % self.index[-1]
              if STEP is None:
                  # prevents an error being thrown in the case of only a single sample being written
                  if STOP != STRT:
                      raw_step = self.index[1] - self.index[0]
                      STEP = fmt % raw_step
          except IndexError:
              pass
  
          self.well["STRT"].value = STRT
          self.well["STOP"].value = STOP
          self.well["STEP"].value = STEP
  
      def update_units_from_index_curve(self):
          """Align STRT/STOP/STEP header item units with the index curve's units."""
          # Check units
          if self.curves and self.curves[0].unit:
              unit = self.curves[0].unit
          else:
              unit = self.well["STRT"].unit
          self.well["STRT"].unit = unit
          self.well["STOP"].unit = unit
          self.well["STEP"].unit = unit
          # Check that curves exists to avoid throwing an expection.
          # to write to an non-existant object.
          if self.curves:
              self.curves[0].unit = unit
  
      def write(self, file_ref, **kwargs):
          """Write LAS file to disk.
  
          Arguments:
              file_ref (open file-like object or str): a file-like object opening
                  for writing, or a filename.
  
          All ``**kwargs`` are passed to :func:`lasio.writer.write` -- please
          check the docstring of that function for more keyword arguments you can
          use here!
  
          Examples:
  
              >>> import lasio
              >>> las = lasio.read("tests/examples/sample.las")
              >>> with open('test_output.las', mode='w') as f:
              ...     las.write(f, version=2.0)   # <-- this method
  
          """
          opened_file = False
          if isinstance(file_ref, basestring) and not hasattr(file_ref, "write"):
              opened_file = True
              file_ref = open(file_ref, "w")
          writer.write(self, file_ref, **kwargs)
          if opened_file:
              file_ref.close()
  
      def to_excel(self, filename):
          """Export LAS file to a Microsoft Excel workbook.
  
          This function will raise an :exc:`ImportError` if ``openpyxl`` is not
          installed.
  
          Arguments:
              filename (str)
  
          """
  
          try:
              import openpyxl
          except ImportError:
              pass
          else:
              from .excel import ExcelConverter
  
          from . import excel
  
          converter = excel.ExcelConverter(self)
          converter.write(filename)
  
      def to_csv(self, file_ref, mnemonics=True, units=True, units_loc="line", **kwargs):
          """Export to a CSV file.
  
          Arguments:
              file_ref (open file-like object or str): a file-like object opening
                  for writing, or a filename.
  
          Keyword Arguments:
              mnemonics (list, True, False): write mnemonics as a header line at the
                  start. If list, use the supplied items as mnemonics. If True,
                  use the curve mnemonics.
              units (list, True, False): as for mnemonics.
              units_loc (str or None): either 'line', '[]' or '()'. 'line' will put
                  units on the line following the mnemonics (good for WellCAD).
                  '[]' and '()' will put the units in either brackets or
                  parentheses following the mnemonics, on the single header line
                  (better for Excel)
              **kwargs: passed to :class:`csv.writer`. Note that if
                  ``lineterminator`` is **not** specified here
  
  [... the rest is omitted ...]
files: lasio/las.py
files: from __future__ import print_function
  
  try:  # will work in Python 3
      from collections.abc import Sequence
  except ImportError:  # Support Python 2.7
      from collections import Sequence
  
  import csv
  import json
  import logging
  import re
  import sys
  import traceback
  
  # get basestring in py3
  
  try:
      unicode = unicode
  except NameError:
      # 'unicode' is undefined, must be Python 3
      unicode = str
      basestring = (str, bytes)
  else:
      # 'unicode' exists, must be Python 2
      bytes = str
      basestring = basestring
  
  # Required third-party packages available on PyPi:
  
  import numpy as np
  
  # internal lasio imports
  
  from . import exceptions
  from .las_items import HeaderItem, CurveItem, SectionItems, OrderedDict
  from . import defaults
  from . import reader
  from . import writer
  
  logger = logging.getLogger(__name__)
  
  
  class LASFile(object):
  
      """LAS file object.
  
      Keyword Arguments:
          file_ref (file-like object, str): either a filename, an open file
              object, or a string containing the contents of a file.
  
      See these routines for additional keyword arguments you can use when
      reading in a LAS file:
  
      * :func:`lasio.reader.open_with_codecs` - manage issues relate to character
        encodings
      * :meth:`lasio.LASFile.read` - control how NULL values and errors are
        handled during parsing
  
      Attributes:
          encoding (str or None): the character encoding used when reading the
              file in from disk
  
      """
  
      def __init__(self, file_ref=None, **read_kwargs):
          super(LASFile, self).__init__()
          self._text = ""
          self.index_unit = None
          self.index_initial = None
          default_items = defaults.get_default_items()
          self.sections = {
              "Version": default_items["Version"],
              "Well": default_items["Well"],
              "Curves": default_items["Curves"],
              "Parameter": default_items["Parameter"],
              "Other": str(default_items["Other"]),
          }
  
          if not (file_ref is None):
              self.read(file_ref, **read_kwargs)
  
      def read(
          self,
          file_ref,
          ignore_header_errors=False,
          ignore_comments=("#",),
          ignore_data_comments="#",
          mnemonic_case="upper",
          ignore_data=False,
          engine="numpy",
          use_normal_engine_for_wrapped=True,
          read_policy="default",
          null_policy="strict",
          index_unit=None,
          dtypes="auto",
          **kwargs
      ):
          """Read a LAS file.
  
          Arguments:
              file_ref (file-like object, str): either a filename, an open file
                  object, or a string containing the contents of a file.
  
          Keyword Arguments:
              ignore_header_errors (bool): ignore LASHeaderErrors (False by
                  default)
              ignore_comments (sequence/str): ignore lines beginning with these
                  characters e.g. ``("#", '"')`` in header sections.
              ignore_data_comments (str): ignore lines beginning with this
                  character in data sections only.
              mnemonic_case (str): 'preserve': keep the case of HeaderItem mnemonics
                                   'upper': convert all HeaderItem mnemonics to uppercase
                                   'lower': convert all HeaderItem mnemonics to lowercase
              ignore_data (bool): if True, do not read in any of the actual data,
                  just the header metadata. False by default.
              engine (str): "normal": parse data section with normal Python reader
                  (quite slow); "numpy": parse data section with `numpy.genfromtxt` (fast).
                  By default the engine is "numpy".
              use_normal_engine_for_wrapped (bool): if header metadata indicates that
                  the file is wrapped, always use the 'normal' engine. Default is True.
                  The only reason you should use False is if speed is a very high priority
                  and you had files with metadata that incorrectly indicates they are
                  wrapped.
              read_policy (): TODO
              null_policy (str or list): see
                  http://lasio.readthedocs.io/en/latest/data-section.html#handling-invalid-data-indicators-automatically
              index_unit (str): Optionally force-set the index curve's unit to "m" or "ft"
              dtypes ("auto", dict or list): specify the data types for each curve in the
                  ~ASCII data section. If "auto", each curve will be converted to floats if
                  possible and remain as str if not. If a dict you can specify only the
                  curve mnemonics you want to convert as a key. If a list, please specify
                  data types for each curve in order. Note that the conversion currently
                  only occurs via numpy.ndarray.astype() and therefore only a few simple
                  casts will work e.g. `int`, `float`, `str`.
              encoding (str): character encoding to open file_ref with, using
                  :func:`io.open` (this is handled by
                  :func:`lasio.reader.open_with_codecs`)
              encoding_errors (str): 'strict', 'replace' (default), 'ignore' - how to
                  handle errors with encodings (see
                  `this section
                  <https://docs.python.org/3/library/codecs.html#codec-base-classes>`__
                  of the standard library's :mod:`codecs` module for more information)
                  (this is handled by :func:`lasio.reader.open_with_codecs`)
              autodetect_encoding (str or bool): default True to use
                  `chardet <https://github.com/chardet/chardet>`__/`cchardet
                  <https://github.com/PyYoshi/cChardet>`__ to detect encoding.
                  Note if set to False several common encodings will be tried but
                  chardet won't be used.
                  (this is handled by :func:`lasio.reader.open_with_codecs`)
              autodetect_encoding_chars (int/None): number of chars to read from LAS
                  file for auto-detection of encoding.
                  (this is handled by :func:`lasio.reader.open_with_codecs`)
  
          """
  
          logger.debug("Reading {}...".format(str(file_ref)))
  
          # Determine which lines to ignore:
          if ignore_comments is None:
              ignore_comments = []
          if isinstance(ignore_comments, str):
              ignore_comments = [ignore_comments]
  
          logger.debug("Ignore header lines beginning with {}".format(ignore_comments))
          logger.debug("Ignore data lines beginning with {}".format(ignore_data_comments))
  
          # Attempt to read file
          file_obj = ""
          try:
              file_obj, self.encoding = reader.open_file(file_ref, **kwargs)
  
              test_lidar = file_obj.read(4)
              if test_lidar == "LASF":
                  raise IOError("This is a LASer file (i.e. LiDAR data), not a Log ASCII Standard file")
              else:
                  file_obj.seek(0)
  
              logger.debug(
                  "Fetching substitutions for read_policy {} and null policy {}".format(
                      read_policy, null_policy
                  )
              )
              provisional_version = 2.0
              provisional_wrapped = "YES"
              provisional_null = None
              provisional_delimiter = "SPACE"
  
              section_positions = reader.find_sections_in_file(file_obj)
              logger.debug("Found {} sections".format(len(section_positions)))
              if len(section_positions) == 0:
                  raise KeyError("No ~ sections found. Is this a LAS file?")
  
              data_section_indices = []
              # This is a transitional data_section_indicies till the las30 data
              # reading can handle 1.2 and 2.0 data, then it will be merged back
              # into data_section_indices
              las3_data_section_indices = []
  
              las3_section_indicators = ["_DATA", "_PARAMETER", "_DEFINITION"]
  
              for i, (k, first_line, last_line, section_title) in enumerate(
                  section_positions
              ):
                  section_type = reader.determine_section_type(section_title)
                  logger.debug(
                      "Parsing {typ} section at lines {first_line}-{last_line} ({k} bytes) {title}".format(
                          typ=section_type,
                          title=section_title,
                          first_line=first_line + 1,
                          last_line=last_line + 1,
                          k=k,
                      )
                  )
  
                  # Read traditional LAS header item section
                  if section_type == "Header items":
                      file_obj.seek(k)
                      sct_items = reader.parse_header_items_section(
                          file_obj,
                          line_nos=(first_line, last_line),
                          version=provisional_version,
                          ignore_header_errors=ignore_header_errors,
                          mnemonic_case=mnemonic_case,
                          ignore_comments=ignore_comments,
                      )
  
                      # Update provisional statuses
                      if "VERS" in sct_items:
                          provisional_version = sct_items.VERS.value
                      if "WRAP" in sct_items:
                          provisional_wrapped = sct_items.WRAP.value
                      if "NULL" in sct_items:
                          provisional_null = sct_items.NULL.value
                      if "DLM" in sct_items:
                          provisional_delimiter = sct_items.DLM.value
  
                      # las3 sections can contain _Data, _Parameter or _Definition
                      las3_section = any(
                          [
                              section_str in section_title[1:].upper()
                              for section_str in las3_section_indicators
                          ]
                      )
  
                      # TODO: Revise so that Version, Well and Parameter(Log_Parameter)
                      # are handled the properly for LAS-1.2, LAS-2.0 LAS-3.0
                      # Set "Curves" for LAS-1.2, LAS-2.0, LAS-3.0
                      if (
                          section_title[1] == "C" and "_" not in section_title
                      ) or "~Log_Definition" in section_title:
                          self.sections["Curves"] = sct_items
                      elif (
                          section_title[1] == "P" and "_" not in section_title
                      ) or "~Log_Parameter" in section_title:
                          self.sections["Parameter"] = sct_items
                      # Set any other LAS3.0  sections
                      elif provisional_version == 3.0 and las3_section:
                          self.sections[section_title[1:]] = sct_items
                      # Set regular sections
                      elif section_title[1] == "V":
                          self.sections["Version"] = sct_items
                      elif section_title[1] == "W":
                          self.sections["Well"] = sct_items
                      else:
                          self.sections[section_title[1:]] = sct_items
  
                  # Read free-text LAS header section
                  elif section_type == "Header (other)":
                      file_obj.seek(k)
                      line_no = first_line
                      contents = []
                      for line in file_obj:
                          if line.startswith("~") and line_no == last_line:
                              break
                          if line.startswith("~"):
                              continue
                          line_no += 1
                          contents.append(line.strip("\n").strip())
                          if line_no == last_line:
                              break
                      sct_contents = "\n".join(contents)
  
                      if section_title[1] == "O":
                          self.sections["Other"] = sct_contents
                      else:
                          self.sections[section_title[1:]] = sct_contents
  
                  elif section_type == "Data":
                      logger.debug("Storing reference and returning later...")
                      data_section_indices.append(i)
  
                  # Initial stub for parsing las3 data. This is probably a
                  # transitional section that will merge with 1.2/2.0 data
                  # parsing once fully functional
                  elif section_type == "Las3_Data":
                      logger.debug("Storing Las3_Data reference and returning later...")
                      las3_data_section_indices.append(i)
  
              line_splitter = reader.define_line_splitter(provisional_delimiter)
  
              if provisional_delimiter == "COMMA":
                  read_policy = "comma-delimiter"
              regexp_subs, value_null_subs, version_NULL = reader.get_substitutions(
                  read_policy, null_policy
              )
  
              if not ignore_data:
  
                  # Override the default "numpy" parser with the 'normal' parser
                  # for these conditions:
                  # - file is wrapped
                  # - null_policy is not "strict"
                  # - dtypes is not "auto". Numpy can handle specified dtypes but
                  #   the performance decays to the 'normal' performance level.
  
                  # normal engine.
                  if provisional_wrapped == "YES" or null_policy != "strict" or dtypes != "auto":
                      if engine != "normal":
                          logger.warning("Only engine='normal' can read wrapped files")
                          if use_normal_engine_for_wrapped:
                              engine = "normal"
  
                  if (
                      len(data_section_indices) == 0
                      and len(las3_data_section_indices) > 0
                  ):
                      data_section_indices = las3_data_section_indices
                  # Check for the number of columns in each data section.
                  for k, first_line, last_line, section_title in [
                      section_positions[i] for i in data_section_indices
                  ]:
                      logger.debug("Reading data section {}".format(section_title))
  
                      file_obj.seek(k)
                      n_columns = reader.inspect_data_section(
                          file_obj,
                          (first_line, last_line),
                          regexp_subs,
                          ignore_comments=ignore_data_comments,
                      )
  
                      # How many curves should the reader attempt to find?
                      reader_n_columns = n_columns
                      if reader_n_columns == -1:
                          reader_n_columns = len(self.curves)
  
                      file_obj.seek(k)
  
                      # Convert dtypes passed as dict into list for all columns
                      # defaulting to float for any not specified.
                      if isinstance(dtypes, dict):
                          dtypes = [dtypes.get(c.mnemonic, float) for c in self.curves]
  
                      # Notes see 2d9e43c3 and e960998f for 'try' background
  
                      # Attempt to read the data section
                      if engine == "numpy":
                          try:
                              curves_data_gen = reader.read_data_section_iterative_numpy_engine(
                                  file_obj, (first_line, last_line)
                              )
                          except KeyboardInterrupt:
                              raise
                          except:
                              try:
                                  file_obj.seek(k)
                                  curves_data_gen = (
                                      reader.read_data_section_iterative_normal_engine(
                                          file_obj,
                                          (first_line, last_line),
                                          regexp_subs,
                                          value_null_subs,
                                          ignore_comments=ignore_data_comments,
                                          n_columns=reader_n_columns,
                                          dtypes=dtypes,
                                          line_splitter=line_splitter,
                                      )
                                  )
                              except KeyboardInterrupt:
                                  raise
                              except:
                                  raise exceptions.LASDataError(
                                      traceback.format_exc()[:-1]
                                      + " in data section beginning line {}".format(i + 1)
                                  )
  
                      if engine == "normal":
                          try:
                              curves_data_gen = (
                                  reader.read_data_section_iterative_normal_engine(
                                      file_obj,
                                      (first_line, last_line),
                                      regexp_subs,
                                      value_null_subs,
                                      ignore_comments=ignore_data_comments,
                                      n_columns=reader_n_columns,
                                      dtypes=dtypes,
                                      line_splitter=line_splitter,
                                  )
                              )
                          except KeyboardInterrupt:
                              raise
                          except:
                              raise exceptions.LASDataError(
                                  traceback.format_exc()[:-1]
                                  + " in data section beginning line {}".format(i + 1)
                              )
  
                      # Assign data to curves.
                      data_assigned_to_curves = {curve_idx: False for curve_idx in range(len(self.curves))}
                      curve_idx = 0
                      curve_length = 0
                      for curve_arr in curves_data_gen:
  
                          # Do not replace nulls in the index curve.
                          if version_NULL and curve_arr.dtype == float and curve_idx != 0:
                              logger.debug(
                                  "Replacing {} with nan in {}-th curve".format(
                                      provisional_null, curve_idx
                                  )
                              )
                              curve_arr[curve_arr == provisional_null] = np.nan
  
                          logger.debug(
                              "Assigning data {} to curve #{}".format(
                                  curve_arr, curve_idx
                              )
                          )
                          if curve_length == 0:
                              curve_length = len(curve_arr)
                          if curve_idx < len(self.curves):
                              self.curves[curve_idx].data = curve_arr
                          else:
                              logger.debug("Creating new curve")
                              curve = CurveItem(mnemonic="", data=curve_arr)
                              self.curves.append(curve)
                          data_assigned_to_curves[curve_idx] = True
                          curve_idx += 1
  
                      # Assign missing data indicators for curves which have no data in the
                      # data section.
                      for curve_idx, flag in data_assigned_to_curves.items():
                          if flag is False:
                              logger.warning(
                                  "Curve #{:.0f} '{:s}' is defined in the ~C section "
                                  "but there is no data in ~A"
                                  .format(curve_idx, self.curves[curve_idx].mnemonic)
                              )
                              self.curves[curve_idx].data = np.empty(curve_length) * np.nan
  
          finally:
              if hasattr(file_obj, "close"):
                  file_obj.close()
  
              # TODO: reimplement these warnings!!
  
              ###### logger.warning("No data section (regexp='~A') found")
              ###### logger.warning("No numerical data found inside ~A section")
  
          # Understand the depth/index unit.
  
          if "m" in str(index_unit):
              index_unit = "m"
  
          if index_unit:
              self.index_unit = index_unit
          else:
              check_units_on = []
              for mnemonic in ("STRT", "STOP", "STEP"):
                  if mnemonic in self.well:
                      check_units_on.append(self.well[mnemonic])
              if len(self.curves) > 0:
                  check_units_on.append(self.curves[0])
              matches = []
              for index_unit, possibilities in defaults.DEPTH_UNITS.items():
                  for check_unit in check_units_on:
                      if any([check_unit.unit == p for p in possibilities]) or any(
                          [check_unit.unit.upper() == p for p in possibilities]
                      ):
                          matches.append(index_unit)
              matches = set(matches)
              if len(matches) == 1:
                  self.index_unit = tuple(matches)[0]
              elif len(matches) == 0:
                  self.index_unit = None
              else:
                  logger.warning("Conflicting index units found: {}".format(matches))
                  self.index_unit = None
  
          if len(self.curves) > 0:
              self.index_initial = self.index.copy()
  
      def update_start_stop_step(self, STRT=None, STOP=None, STEP=None, fmt="%.5f"):
          """Configure or change STRT, STOP, and STEP values on the LASFile object.
  
          Keyword Arguments:
              STRT, STOP, STEP (str, int, float): value to set on the relevant
                  header item in the ~Well section - can be any
                  data type.
              fmt (str): Python format string for formatting the STRT/STOP/STEP
                  value in the situation where any of those keyword arguments
                  are None
  
          If STRT/STOP/STEP are not passed to this method, they will be automatically
          calculated from the index curve.
  
          """
  
          # If we are getting STRT and STOP from the data then format them to a
          # standard precision.
          # If they are passed in with values, don't format them because we
          # assume they are at the user's expected precision.
  
          # If the 'try' fails because self.index doesn't exist or is empty
          # then use the default or parameter values for STRT, STOP, and STEP.
          try:
              if STRT is None:
                  STRT = fmt % self.index[0]
              if STOP is None:
                  STOP = fmt % self.index[-1]
              if STEP is None:
                  # prevents an error being thrown in the case of only a single sample being written
                  if STOP != STRT:
                      raw_step = self.index[1] - self.index[0]
                      STEP = fmt % raw_step
          except IndexError:
              pass
  
          self.well["STRT"].value = STRT
          self.well["STOP"].value = STOP
          self.well["STEP"].value = STEP
  
      def update_units_from_index_curve(self):
          """Align STRT/STOP/STEP header item units with the index curve's units."""
          # Check units
          if self.curves and self.curves[0].unit:
              unit = self.curves[0].unit
          else:
              unit = self.well["STRT"].unit
          self.well["STRT"].unit = unit
          self.well["STOP"].unit = unit
          self.well["STEP"].unit = unit
          # Check that curves exists to avoid throwing an expection.
          # to write to an non-existant object.
          if self.curves:
              self.curves[0].unit = unit
  
      def write(self, file_ref, **kwargs):
          """Write LAS file to disk.
  
          Arguments:
              file_ref (open file-like object or str): a file-like object opening
                  for writing, or a filename.
  
          All ``**kwargs`` are passed to :func:`lasio.writer.write` -- please
          check the docstring of that function for more keyword arguments you can
          use here!
  
          Examples:
  
              >>> import lasio
              >>> las = lasio.read("tests/examples/sample.las")
              >>> with open('test_output.las', mode='w') as f:
              ...     las.write(f, version=2.0)   # <-- this method
  
          """
          opened_file = False
          if isinstance(file_ref, basestring) and not hasattr(file_ref, "write"):
              opened_file = True
              file_ref = open(file_ref, "w")
          writer.write(self, file_ref, **kwargs)
          if opened_file:
              file_ref.close()
  
      def to_excel(self, filename):
          """Export LAS file to a Microsoft Excel workbook.
  
          This function will raise an :exc:`ImportError` if ``openpyxl`` is not
          installed.
  
          Arguments:
              filename (str)
  
          """
  
          try:
              import openpyxl
          except ImportError:
              pass
          else:
              from .excel import ExcelConverter
  
          from . import excel
  
          converter = excel.ExcelConverter(self)
          converter.write(filename)
  
      def to_csv(self, file_ref, mnemonics=True, units=True, units_loc="line", **kwargs):
          """Export to a CSV file.
  
          Arguments:
              file_ref (open file-like object or str): a file-like object opening
                  for writing, or a filename.
  
          Keyword Arguments:
              mnemonics (list, True, False): write mnemonics as a header line at the
                  start. If list, use the supplied items as mnemonics. If True,
                  use the curve mnemonics.
              units (list, True, False): as for mnemonics.
              units_loc (str or None): either 'line', '[]' or '()'. 'line' will put
                  units on the line following the mnemonics (good for WellCAD).
                  '[]' and '()' will put the units in either brackets or
                  parentheses following the mnemonics, on the single header line
                  (better for Excel)
              **kwargs: passed to :class:`csv.writer`. Note that if
                  ``lineterminator`` is **not** specified here, then it
  
  [... the rest is omitted ...]
files: lasio/las.py
files: import codecs
  import io
  import logging
  import os
  import re
  import sys
  import traceback
  import urllib.request
  
  import numpy as np
  
  from . import defaults
  
  # Convoluted import for StringIO in order to support:
  #
  # - Python 3 - io.StringIO
  # - Python 2 (optimized) - cStringIO.StringIO
  # - Python 2 (all) - StringIO.StringIO
  
  try:
      import cStringIO as StringIO
  except ImportError:
      try:  # cStringIO not available on this system
          import StringIO
      except ImportError:  # Python 3
          from io import StringIO
      else:
          from StringIO import StringIO
  else:
      from StringIO import StringIO
  
  from . import exceptions
  from .las_items import HeaderItem, CurveItem, SectionItems, OrderedDict
  
  
  logger = logging.getLogger(__name__)
  
  URL_REGEXP = re.compile(
      r"^(?:http|ftp)s?://"  # http:// or https://
      r"(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}"
      r"\.?|[A-Z0-9-]{2,}\.?)|"  # (cont.) domain...
      r"localhost|"  # localhost...
      r"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"  # ...or ip
      r"(?::\d+)?"  # optional port
      r"(?:/?|[/?]\S+)$",
      re.IGNORECASE,
  )
  
  # sow (Split On Whitespace) regex
  sow_regex = re.compile(r"""([^\s"']+)|"([^"]*)"|'([^']*)'""")
  
  
  def define_line_splitter(provisional_delimiter):
      """Define multiple line splitters
  
      return the one that is right for the data delmiter
  
      """
      sow_regex = re.compile(r"""([^\s"']+)|"([^"]*)"|'([^']*)'""")
  
      def split_on_whitespace(line):
          return sow_regex.findall(line)
  
      def split_on_comma(line):
          return line.split(",")
  
      splitters = {
          "SPACE": split_on_whitespace,
          "COMMA": split_on_comma,
      }
  
      return splitters[provisional_delimiter]
  
  
  def check_for_path_obj(file_ref):
      """Check if file_ref is a pathlib.Path object.
  
      If file_ref is a pathlib.Path object, then return its absolute file
      path as a string so it will get processed as other string filenames.
  
      If pathlib is not available, do nothing and return file_ref.
  
      """
      try:
          from pathlib import Path
      except ImportError:
          return file_ref
  
      if isinstance(file_ref, Path):
          return file_ref.absolute().__str__()
      else:
          return file_ref
  
  
  def open_file(file_ref, **encoding_kwargs):
      """Open a file if necessary.
  
      If ``autodetect_encoding=True`` then either ``cchardet`` or ``chardet``
      needs to be installed, or else an ``ImportError`` will be raised.
  
      Arguments:
          file_ref (file-like object, str): either a filename, an open file
              object, or a string containing the contents of a file.
  
      See :func:`lasio.reader.open_with_codecs` for keyword arguments that can be
      used here.
  
      Returns:
          tuple of an open file-like object, and the encoding that
          was used to decode it (if it were read from disk).
  
      """
  
      file_ref = check_for_path_obj(file_ref)
  
      encoding = None
      if isinstance(file_ref, str):  # file_ref != file-like object, so what is it?
          lines = file_ref.splitlines()
          first_line = lines[0]
          if URL_REGEXP.match(first_line):  # it's a URL
              logger.info("Loading URL {}".format(first_line))
  
              response = urllib.request.urlopen(file_ref)
              if response.headers.get_content_charset() is None:
                  if "encoding" in encoding_kwargs:
                      encoding = encoding_kwargs["encoding"]
                  else:
                      encoding = "utf-8"
              else:
                  encoding = response.headers.get_content_charset()
              # newline=None causes StringIO to use universal-newline:
              # Lines in the input can end in '\n', '\r', or '\r\n', and these are
              # translated into '\n' before being returned to the caller.
              file_ref = StringIO(response.read().decode(encoding), newline=None)
              logger.debug("Retrieved data decoded via {}".format(encoding))
          elif len(lines) > 1:  # it's LAS data as a string.
              file_ref = StringIO(file_ref)
          else:  # it must be a filename
              file_ref, encoding = open_with_codecs(first_line, **encoding_kwargs)
      return file_ref, encoding
  
  
  def open_with_codecs(
      filename,
      encoding=None,
      encoding_errors="replace",
      autodetect_encoding=True,
      autodetect_encoding_chars=4000,
  ):
      """
      Read Unicode data from file.
  
      Arguments:
          filename (str): path to file
  
      Keyword Arguments:
          encoding (str): character encoding to open file_ref with, using
              :func:`io.open`.
          encoding_errors (str): 'strict', 'replace' (default), 'ignore' - how to
              handle errors with encodings (see
              `this section
              <https://docs.python.org/3/library/codecs.html#codec-base-classes>`__
              of the standard library's :mod:`codecs` module for more information)
          autodetect_encoding (str or bool): default True to use
              `chardet <https://github.com/chardet/chardet>`__/`cchardet
              <https://github.com/PyYoshi/cChardet>`__ to detect encoding.
              Note if set to False several common encodings will be tried but
              chardet won't be used.
          autodetect_encoding_chars (int/None): number of chars to read from LAS
              file for auto-detection of encoding.
  
      Returns:
          a unicode or string object
  
      This function is called by :func:`lasio.reader.open_file`.
  
      """
      if autodetect_encoding_chars:
          nbytes = int(autodetect_encoding_chars)
      else:
          nbytes = None
  
      # Forget [c]chardet - if we can locate the BOM we just assume that's correct.
      nbytes_test = min(32, os.path.getsize(filename))
      with open(filename, mode="rb") as test:
          raw = test.read(nbytes_test)
      if raw.startswith(codecs.BOM_UTF8):
          encoding = "utf-8-sig"
          autodetect_encoding = False
  
      # If BOM wasn't found...
      if (autodetect_encoding) and (not encoding):
          with open(filename, mode="rb") as test:
              if nbytes is None:
                  raw = test.read()
              else:
                  raw = test.read(nbytes)
          encoding = get_encoding(autodetect_encoding, raw)
          autodetect_encoding = False
  
      # Or if no BOM found & chardet not installed
      if (not autodetect_encoding) and (not encoding):
          encoding = adhoc_test_encoding(filename)
          if encoding:
              logger.info(
                  "{} was found by ad hoc to work but note it might not"
                  " be the correct encoding".format(encoding)
              )
  
      # Now open and return the file-like object
      logger.info(
          'Opening {} as {} and treating errors with "{}"'.format(
              filename, encoding, encoding_errors
          )
      )
      file_obj = io.open(filename, mode="r", encoding=encoding, errors=encoding_errors)
      return file_obj, encoding
  
  
  def adhoc_test_encoding(filename):
      test_encodings = ["ascii", "windows-1252", "latin-1"]
      for i in test_encodings:
          encoding = i
          with io.open(filename, mode="r", encoding=encoding) as f:
              try:
                  f.readline()
                  break
              except UnicodeDecodeError:
                  logger.debug("{} tested, raised UnicodeDecodeError".format(i))
                  pass
              encoding = None
      return encoding
  
  
  def get_encoding(auto, raw):
      """
      Automatically detect character encoding.
  
      Arguments:
          auto (str): auto-detection of character encoding - can be either
              'chardet', 'cchardet', False, or True (the latter will pick the
              fastest available option)
          raw (bytes): array of bytes to detect from
  
      Returns:
          A string specifying the character encoding.
  
      """
      if auto is True:
          try:
              import cchardet as chardet
          except ImportError:
              try:
                  import chardet
              except ImportError:
                  logger.debug(
                      "chardet or cchardet is recommended for automatic"
                      " detection of character encodings. Instead trying some"
                      " common encodings."
                  )
                  return None
              else:
                  logger.debug("get_encoding Using chardet")
                  method = "chardet"
          else:
              logger.debug("get_encoding Using cchardet")
              method = "cchardet"
      elif auto.lower() == "chardet":
          import chardet
  
          logger.debug("get_encoding Using chardet")
          method = "chardet"
      elif auto.lower() == "cchardet":
          import cchardet as chardet
  
          logger.debug("get_encoding Using cchardet")
          method = "cchardet"
      result = chardet.detect(raw)
      logger.debug(
          "{} method detected encoding of {} at confidence {}".format(
              method, result["encoding"], result["confidence"]
          )
      )
      return result["encoding"]
  
  
  def find_sections_in_file(file_obj):
      """Find LAS sections in a file.
  
      Returns: a list of lists *(k, first_line_no, last_line_no, line]*.
          *file_pos* is the position in the *file_obj* in bytes,
          *first_line_no* is the first line number of the section (starting
          from zero), and *line* is the contents of the section title/definition
          i.e. beginning with ``~`` but stripped of beginning or ending whitespace
          or line breaks.
  
      """
      file_pos = int(file_obj.tell())
      starts = []
      ends = []
      line_no = 0
      line = file_obj.readline()
      # for i, line in enumerate(file_obj):
      while line:
          sline = line.strip().strip("\n")
          if sline.startswith("~"):
              starts.append((file_pos, line_no, sline))
              if len(starts) > 1:
                  ends.append(line_no - 1)
          file_pos = int(file_obj.tell())
          line = file_obj.readline()
          line_no = line_no + 1
  
      ends.append(line_no)
      section_positions = []
      for j, (file_pos, first_line_no, sline) in enumerate(starts):
          section_positions.append((file_pos, first_line_no, ends[j], sline))
      return section_positions
  
  
  def determine_section_type(section_title):
      """Return the type of the LAS section based on its title
  
          >>> determine_section_type("~Curves Section")
          "Header"
          >>> determine_section_type("~ASCII")
          "Data"
  
      Returns: bool
  
      """
      stitle = section_title.strip().strip("\n")
      # '~Log_Data' is a LAS-3.0 equivalent for the ~ASCII data section
      if stitle[:2] == "~A" or "~Log_Data" in stitle:
          return "Data"
      elif stitle[:2] == "~O":
          return "Header (other)"
      # This is las3 transitional code till data parsing is robust for ~A and
      # '_Data' sections
      elif re.search("_Data", stitle):
          return "Las3_Data"
      else:
          return "Header items"
  
  
  def inspect_data_section(file_obj, line_nos, regexp_subs, ignore_data_comments="#"):
      """Determine how many columns there are in the data section.
  
      Arguments:
          file_obj: file-like object open for reading at the beginning of the section
          line_nos (tuple): the first and last line no of the section to read
          regexp_subs (list): each item should be a tuple of the pattern and
              substitution string for a call to re.sub() on each line of the
              data section. See defaults.py READ_SUBS and NULL_SUBS for examples.
          ignore_data_comments (str): lines beginning with this character will be ignored
  
      Returns: integer number of columns or -1 where they are different.
  
      """
  
      line_no = line_nos[0]
      title_line = file_obj.readline()
  
      item_counts = []
  
      for i, line in enumerate(file_obj):
          line_no = line_no + 1
          line = line.strip("\n").strip()
          if line.strip().startswith(ignore_data_comments):
              continue
          else:
              for pattern, sub_str in regexp_subs:
                  line = re.sub(pattern, sub_str, line)
              # split line and count number of elements
              n_items = len(["".join(t) for t in sow_regex.findall(line)])
              logger.trace_lasio(
                  "Line {}: {} items counted in '{}'".format(line_no + 1, n_items, line)
              )
              item_counts.append(n_items)
              if (line_no == line_nos[1]) or (i >= 20):
                  break
  
      try:
          assert len(set(item_counts)) == 1
      except AssertionError:
          logger.debug("Inconsistent number of columns {}".format(item_counts))
          return -1
      else:
          logger.debug("Consistently found {} columns".format(item_counts[0]))
          return item_counts[0]
  
  
  def read_data_section_iterative_normal_engine(
      file_obj,
      line_nos,
      regexp_subs,
      value_null_subs,
      ignore_data_comments,
      n_columns,
      dtypes,
      line_splitter,
  ):
      """Read data section into memory.
  
      Arguments:
          file_obj: file-like object open for reading at the beginning of the section
          line_nos (tuple): the first and last line no of the section to read
          regexp_subs (list): each item should be a tuple of the pattern and
              substitution string for a call to re.sub() on each line of the
              data section. See defaults.py READ_SUBS and NULL_SUBS for examples.
          value_null_subs (list): list of numerical values to be replaced by
              numpy.nan values.
          ignore_data_comments (str): lines beginning with this character will be ignored
          n_columns (int): expected number of columns
          dtypes (list, "auto", False): list of expected data types for each column,
              (each data type can be specified as e.g. `int`,
              `float`, `str`, `datetime`). If you specify 'auto', then this function
              will attempt to convert each column to a float and if that fails,
              the column will be returned as a string. If you specify False, no
              conversion of data types will be attempt at all.
          line_splitter (function): This function is dynamically configured to
              split data lines on the configured delimiter
  
      Returns: generator which yields the data as a 1D ndarray for each column at a time.
  
      """
      logger.debug(
          "Attempting to read {} columns between lines {}".format(n_columns, line_nos)
      )
  
      title = file_obj.readline()
  
      def items(f, start_line_no, end_line_no):
          for line_no, line in enumerate(f, start=start_line_no+1):
              line = line.strip("\n").strip()
              if line.startswith(ignore_data_comments):
                  continue
              else:
                  for pattern, sub_str in regexp_subs:
                      line = re.sub(pattern, sub_str, line)
                  line = line.replace(chr(26), "")
                  if len(line) == 0:
                      continue
  
                  # for item in split_on_whitespace(line, sow_regex):
                  # for item in ["".join(t) for t in sow_regex.findall(line)]:
                  for item in ["".join(t) for t in line_splitter(line)]:
                      try:
                          yield np.float64(item)
                      except ValueError:
                          yield item
                  if line_no == end_line_no:
                      break
  
      logger.debug("Reading complete data section...")
      array = np.array(
          [i for i in items(file_obj, start_line_no=line_nos[0], end_line_no=line_nos[1])]
      )
      for value in value_null_subs:
          array[array == value] = np.nan
  
      logger.debug("Read {} items in data section".format(len(array)))
  
      # Cater for situations where the data section is empty.
      if len(array) == 0:
          logger.warning("Data section is empty therefore setting n_columns to zero")
          n_columns = 0
  
      # Re-shape the 1D array to a 2D array.
      if n_columns > 0:
          logger.debug("Attempt re-shape to {} columns".format(n_columns))
          try:
              array = np.reshape(array, (-1, n_columns))
          except ValueError as exception:
              error_message = "Cannot reshape ~A data size {0} into {1} columns".format(
                  array.shape, n_columns
              )
              if sys.version_info.major < 3:
                  exception.message = error_message
                  raise exception
              else:
                  raise ValueError(error_message).with_traceback(exception.__traceback__)
  
      # Identify how many columns have actually been found.
      if len(array.shape) < 2:
          arr_n_cols = 0
      else:
          arr_n_cols = array.shape[1]
  
      # Identify what the appropriate data types should be for each column based on the first
      # row of the data.
      if dtypes == "auto":
          if len(array) > 0:
              dtypes = identify_dtypes_from_data(array[0, :])
          else:
              dtypes = []
      elif dtypes is False:
          dtypes = [str for n in range(arr_n_cols)]
  
      # Iterate over each column, convert to the appropriate dtype (if possible)
      # and then yield the data column.
      for col_idx in range(arr_n_cols):
          curve_arr = array[:, col_idx]
          curve_dtype = dtypes[col_idx]
          try:
              curve_arr = curve_arr.astype(curve_dtype, copy=False)
          except ValueError:
              logger.warning(
                  "Could not convert curve #{} to {}".format(col_idx, curve_dtype)
              )
          else:
              logger.debug(
                  "Converted curve {} to {} ({})".format(col_idx, curve_dtype, curve_arr)
              )
          yield curve_arr
  
  
  def identify_dtypes_from_data(row):
      """Identify which columns should be 'str' and which 'float'.
  
      Args:
          row (1D ndarray): first row of data section
  
      Returns: list of [float, float, str, ...] etc
  
      """
      logger.debug("Creating auto dtype spec from first line of data array")
      dtypes_list = []
      for i, value in enumerate(row):
          try:
              value_converted = float(value)
          except:
              dtypes_list.append(str)
          else:
              dtypes_list.append(float)
          logger.debug(
              "Column {}: value {} -> dtype {}".format(i, value, dtypes_list[-1])
          )
      return dtypes_list
  
  
  def read_data_section_iterative_numpy_engine(file_obj, line_nos):
      """Read data section into memory.
  
      Arguments:
          file_obj: file-like object open for reading at the beginning of the section
          line_nos (tuple): the first and last line no of the section to read
  
  
      Returns:
          A numpy ndarray.
      """
  
      first_line = line_nos[0] + 1
      last_line = line_nos[1]
      max_rows = last_line - first_line
  
      file_obj.seek(0)
  
      # unpack=True transforms the data from an array of rows to an array of columns.
      # loose=False will throw an error on non-numerical data, which then sends the 
      # parsing to the 'normal' parser.
      array = np.genfromtxt(
          file_obj, skip_header=first_line, max_rows=max_rows, names=None, unpack=True, loose=False
      )
  
      # If there is only one data row, np.genfromtxt treats it as one array of
      # individual values. Lasio needs a array of arrays. This if statement
      # converts the single line data array to an array of arrays(column data).
      if len(array.shape) == 1:
          arr_len = array.shape[0]
          array = array.reshape(arr_len,1)
  
      return array
  
  
  def get_substitutions(read_policy, null_policy):
      """Parse read and null policy definitions into a list of regexp and value
      substitutions.
  
      Arguments:
          read_policy (str, list, or substitution): either (1) a string defined in
              defaults.READ_POLICIES; (2) a list of substitutions as defined by
              the keys of defaults.READ_SUBS; or (3) a list of actual substitutions
              similar to the values of defaults.READ_SUBS. You can mix (2) and (3)
              together if you want.
          null_policy (str, list, or sub): as for read_policy but for
              defaults.NULL_POLICIES and defaults.NULL_SUBS
  
      Returns:
          regexp_subs, value_null_subs, version_NULL - two lists and a bool.
          The first list is pairs of regexp patterns and substrs, and the second
          list is just a list of floats or integers. The bool is whether or not
          'NULL' was located as a substitution.
  
      """
      regexp_subs = []
      numerical_subs = []
      version_NULL = False
  
      for policy_typ, policy, policy_subs, subs in (
          ("read", read_policy, defaults.READ_POLICIES, defaults.READ_SUBS),
          ("null", null_policy, defaults.NULL_POLICIES, defaults.NULL_SUBS),
      ):
          try:
              is_policy = policy in policy_subs
          except TypeError:
              is_policy = False
          if is_policy:
              logger.debug('using {} policy of "{}"'.format(policy_typ, policy))
              all_subs = []
              for sub in policy_subs[policy]:
                  logger.debug("adding substitution {}".format(sub))
                  if sub in subs:
                      all_subs += subs[sub]
                  if sub == "NULL":
                      logger.debug("located substitution for LAS.version.NULL as True")
                      version_NULL = True
          else:
              all_subs = []
              for item in policy:
                  if item in subs:
                      all_subs += subs[item]
                      if item == "NULL":
                          logger.debug(
                              "located substitution for LAS.version.NULL as True"
                          )
                          version_NULL = True
                  else:
                      all_subs.append(item)
          for item in all_subs:
              try:
                  iter(item)
              except TypeError:
                  logger.debug("added numerical substitution: {}".format(item))
                  numerical_subs.append(item)
              else:
                  logger.debug(
                      'added regexp substitution: pattern={} substr="{}"'.format(
                          item[0], item[1]
                      )
                  )
                  regexp_subs.append(item)
      numerical_subs = [n for n in numerical_subs if not n is None]
  
      return regexp_subs, numerical_subs, version_N
  
  [... the rest is omitted ...]
files: lasio/reader.py
files: import codecs
  import io
  import logging
  import os
  import re
  import sys
  import traceback
  import urllib.request
  
  import numpy as np
  
  from . import defaults
  
  # Convoluted import for StringIO in order to support:
  #
  # - Python 3 - io.StringIO
  # - Python 2 (optimized) - cStringIO.StringIO
  # - Python 2 (all) - StringIO.StringIO
  
  try:
      import cStringIO as StringIO
  except ImportError:
      try:  # cStringIO not available on this system
          import StringIO
      except ImportError:  # Python 3
          from io import StringIO
      else:
          from StringIO import StringIO
  else:
      from StringIO import StringIO
  
  from . import exceptions
  from .las_items import HeaderItem, CurveItem, SectionItems, OrderedDict
  
  
  logger = logging.getLogger(__name__)
  
  URL_REGEXP = re.compile(
      r"^(?:http|ftp)s?://"  # http:// or https://
      r"(?:(?:[A-Z0-9](?:[A-Z0-9-]{0,61}[A-Z0-9])?\.)+(?:[A-Z]{2,6}"
      r"\.?|[A-Z0-9-]{2,}\.?)|"  # (cont.) domain...
      r"localhost|"  # localhost...
      r"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"  # ...or ip
      r"(?::\d+)?"  # optional port
      r"(?:/?|[/?]\S+)$",
      re.IGNORECASE,
  )
  
  # sow (Split On Whitespace) regex
  sow_regex = re.compile(r"""([^\s"']+)|"([^"]*)"|'([^']*)'""")
  
  
  def define_line_splitter(provisional_delimiter):
      """Define multiple line splitters
  
      return the one that is right for the data delmiter
  
      """
      sow_regex = re.compile(r"""([^\s"']+)|"([^"]*)"|'([^']*)'""")
  
      def split_on_whitespace(line):
          return sow_regex.findall(line)
  
      def split_on_comma(line):
          return line.split(",")
  
      splitters = {
          "SPACE": split_on_whitespace,
          "COMMA": split_on_comma,
      }
  
      return splitters[provisional_delimiter]
  
  
  def check_for_path_obj(file_ref):
      """Check if file_ref is a pathlib.Path object.
  
      If file_ref is a pathlib.Path object, then return its absolute file
      path as a string so it will get processed as other string filenames.
  
      If pathlib is not available, do nothing and return file_ref.
  
      """
      try:
          from pathlib import Path
      except ImportError:
          return file_ref
  
      if isinstance(file_ref, Path):
          return file_ref.absolute().__str__()
      else:
          return file_ref
  
  
  def open_file(file_ref, **encoding_kwargs):
      """Open a file if necessary.
  
      If ``autodetect_encoding=True`` then either ``cchardet`` or ``chardet``
      needs to be installed, or else an ``ImportError`` will be raised.
  
      Arguments:
          file_ref (file-like object, str): either a filename, an open file
              object, or a string containing the contents of a file.
  
      See :func:`lasio.reader.open_with_codecs` for keyword arguments that can be
      used here.
  
      Returns:
          tuple of an open file-like object, and the encoding that
          was used to decode it (if it were read from disk).
  
      """
  
      file_ref = check_for_path_obj(file_ref)
  
      encoding = None
      if isinstance(file_ref, str):  # file_ref != file-like object, so what is it?
          lines = file_ref.splitlines()
          first_line = lines[0]
          if URL_REGEXP.match(first_line):  # it's a URL
              logger.info("Loading URL {}".format(first_line))
  
              response = urllib.request.urlopen(file_ref)
              if response.headers.get_content_charset() is None:
                  if "encoding" in encoding_kwargs:
                      encoding = encoding_kwargs["encoding"]
                  else:
                      encoding = "utf-8"
              else:
                  encoding = response.headers.get_content_charset()
              # newline=None causes StringIO to use universal-newline:
              # Lines in the input can end in '\n', '\r', or '\r\n', and these are
              # translated into '\n' before being returned to the caller.
              file_ref = StringIO(response.read().decode(encoding), newline=None)
              logger.debug("Retrieved data decoded via {}".format(encoding))
          elif len(lines) > 1:  # it's LAS data as a string.
              file_ref = StringIO(file_ref)
          else:  # it must be a filename
              file_ref, encoding = open_with_codecs(first_line, **encoding_kwargs)
      return file_ref, encoding
  
  
  def open_with_codecs(
      filename,
      encoding=None,
      encoding_errors="replace",
      autodetect_encoding=True,
      autodetect_encoding_chars=4000,
  ):
      """
      Read Unicode data from file.
  
      Arguments:
          filename (str): path to file
  
      Keyword Arguments:
          encoding (str): character encoding to open file_ref with, using
              :func:`io.open`.
          encoding_errors (str): 'strict', 'replace' (default), 'ignore' - how to
              handle errors with encodings (see
              `this section
              <https://docs.python.org/3/library/codecs.html#codec-base-classes>`__
              of the standard library's :mod:`codecs` module for more information)
          autodetect_encoding (str or bool): default True to use
              `chardet <https://github.com/chardet/chardet>`__/`cchardet
              <https://github.com/PyYoshi/cChardet>`__ to detect encoding.
              Note if set to False several common encodings will be tried but
              chardet won't be used.
          autodetect_encoding_chars (int/None): number of chars to read from LAS
              file for auto-detection of encoding.
  
      Returns:
          a unicode or string object
  
      This function is called by :func:`lasio.reader.open_file`.
  
      """
      if autodetect_encoding_chars:
          nbytes = int(autodetect_encoding_chars)
      else:
          nbytes = None
  
      # Forget [c]chardet - if we can locate the BOM we just assume that's correct.
      nbytes_test = min(32, os.path.getsize(filename))
      with open(filename, mode="rb") as test:
          raw = test.read(nbytes_test)
      if raw.startswith(codecs.BOM_UTF8):
          encoding = "utf-8-sig"
          autodetect_encoding = False
  
      # If BOM wasn't found...
      if (autodetect_encoding) and (not encoding):
          with open(filename, mode="rb") as test:
              if nbytes is None:
                  raw = test.read()
              else:
                  raw = test.read(nbytes)
          encoding = get_encoding(autodetect_encoding, raw)
          autodetect_encoding = False
  
      # Or if no BOM found & chardet not installed
      if (not autodetect_encoding) and (not encoding):
          encoding = adhoc_test_encoding(filename)
          if encoding:
              logger.info(
                  "{} was found by ad hoc to work but note it might not"
                  " be the correct encoding".format(encoding)
              )
  
      # Now open and return the file-like object
      logger.info(
          'Opening {} as {} and treating errors with "{}"'.format(
              filename, encoding, encoding_errors
          )
      )
      file_obj = io.open(filename, mode="r", encoding=encoding, errors=encoding_errors)
      return file_obj, encoding
  
  
  def adhoc_test_encoding(filename):
      test_encodings = ["ascii", "windows-1252", "latin-1"]
      for i in test_encodings:
          encoding = i
          with io.open(filename, mode="r", encoding=encoding) as f:
              try:
                  f.readline()
                  break
              except UnicodeDecodeError:
                  logger.debug("{} tested, raised UnicodeDecodeError".format(i))
                  pass
              encoding = None
      return encoding
  
  
  def get_encoding(auto, raw):
      """
      Automatically detect character encoding.
  
      Arguments:
          auto (str): auto-detection of character encoding - can be either
              'chardet', 'cchardet', False, or True (the latter will pick the
              fastest available option)
          raw (bytes): array of bytes to detect from
  
      Returns:
          A string specifying the character encoding.
  
      """
      if auto is True:
          try:
              import cchardet as chardet
          except ImportError:
              try:
                  import chardet
              except ImportError:
                  logger.debug(
                      "chardet or cchardet is recommended for automatic"
                      " detection of character encodings. Instead trying some"
                      " common encodings."
                  )
                  return None
              else:
                  logger.debug("get_encoding Using chardet")
                  method = "chardet"
          else:
              logger.debug("get_encoding Using cchardet")
              method = "cchardet"
      elif auto.lower() == "chardet":
          import chardet
  
          logger.debug("get_encoding Using chardet")
          method = "chardet"
      elif auto.lower() == "cchardet":
          import cchardet as chardet
  
          logger.debug("get_encoding Using cchardet")
          method = "cchardet"
      result = chardet.detect(raw)
      logger.debug(
          "{} method detected encoding of {} at confidence {}".format(
              method, result["encoding"], result["confidence"]
          )
      )
      return result["encoding"]
  
  
  def find_sections_in_file(file_obj):
      """Find LAS sections in a file.
  
      Returns: a list of lists *(k, first_line_no, last_line_no, line]*.
          *file_pos* is the position in the *file_obj* in bytes,
          *first_line_no* is the first line number of the section (starting
          from zero), and *line* is the contents of the section title/definition
          i.e. beginning with ``~`` but stripped of beginning or ending whitespace
          or line breaks.
  
      """
      file_pos = int(file_obj.tell())
      starts = []
      ends = []
      line_no = 0
      line = file_obj.readline()
      # for i, line in enumerate(file_obj):
      while line:
          sline = line.strip().strip("\n")
          if sline.startswith("~"):
              starts.append((file_pos, line_no, sline))
              if len(starts) > 1:
                  ends.append(line_no - 1)
          file_pos = int(file_obj.tell())
          line = file_obj.readline()
          line_no = line_no + 1
  
      ends.append(line_no)
      section_positions = []
      for j, (file_pos, first_line_no, sline) in enumerate(starts):
          section_positions.append((file_pos, first_line_no, ends[j], sline))
      return section_positions
  
  
  def determine_section_type(section_title):
      """Return the type of the LAS section based on its title
  
          >>> determine_section_type("~Curves Section")
          "Header"
          >>> determine_section_type("~ASCII")
          "Data"
  
      Returns: bool
  
      """
      stitle = section_title.strip().strip("\n")
      # '~Log_Data' is a LAS-3.0 equivalent for the ~ASCII data section
      if stitle[:2] == "~A" or "~Log_Data" in stitle:
          return "Data"
      elif stitle[:2] == "~O":
          return "Header (other)"
      # This is las3 transitional code till data parsing is robust for ~A and
      # '_Data' sections
      elif re.search("_Data", stitle):
          return "Las3_Data"
      else:
          return "Header items"
  
  
  def inspect_data_section(file_obj, line_nos, regexp_subs, ignore_comments="#"):
      """Determine how many columns there are in the data section.
  
      Arguments:
          file_obj: file-like object open for reading at the beginning of the section
          line_nos (tuple): the first and last line no of the section to read
          regexp_subs (list): each item should be a tuple of the pattern and
              substitution string for a call to re.sub() on each line of the
              data section. See defaults.py READ_SUBS and NULL_SUBS for examples.
          ignore_comments (str): lines beginning with this character will be ignored
  
      Returns: integer number of columns or -1 where they are different.
  
      """
  
      line_no = line_nos[0]
      title_line = file_obj.readline()
  
      item_counts = []
  
      for i, line in enumerate(file_obj):
          line_no = line_no + 1
          line = line.strip("\n").strip()
          if line.strip().startswith(ignore_comments):
              continue
          else:
              for pattern, sub_str in regexp_subs:
                  line = re.sub(pattern, sub_str, line)
              # split line and count number of elements
              n_items = len(["".join(t) for t in sow_regex.findall(line)])
              logger.trace_lasio(
                  "Line {}: {} items counted in '{}'".format(line_no + 1, n_items, line)
              )
              item_counts.append(n_items)
              if (line_no == line_nos[1]) or (i >= 20):
                  break
  
      try:
          assert len(set(item_counts)) == 1
      except AssertionError:
          logger.debug("Inconsistent number of columns {}".format(item_counts))
          return -1
      else:
          logger.debug("Consistently found {} columns".format(item_counts[0]))
          return item_counts[0]
  
  
  def read_data_section_iterative_normal_engine(
      file_obj,
      line_nos,
      regexp_subs,
      value_null_subs,
      ignore_comments,
      n_columns,
      dtypes,
      line_splitter,
  ):
      """Read data section into memory.
  
      Arguments:
          file_obj: file-like object open for reading at the beginning of the section
          line_nos (tuple): the first and last line no of the section to read
          regexp_subs (list): each item should be a tuple of the pattern and
              substitution string for a call to re.sub() on each line of the
              data section. See defaults.py READ_SUBS and NULL_SUBS for examples.
          value_null_subs (list): list of numerical values to be replaced by
              numpy.nan values.
          ignore_comments (str): lines beginning with this character will be ignored
          n_columns (int): expected number of columns
          dtypes (list, "auto", False): list of expected data types for each column,
              (each data type can be specified as e.g. `int`,
              `float`, `str`, `datetime`). If you specify 'auto', then this function
              will attempt to convert each column to a float and if that fails,
              the column will be returned as a string. If you specify False, no
              conversion of data types will be attempt at all.
          line_splitter (function): This function is dynamically configured to
              split data lines on the configured delimiter
  
      Returns: generator which yields the data as a 1D ndarray for each column at a time.
  
      """
      logger.debug(
          "Attempting to read {} columns between lines {}".format(n_columns, line_nos)
      )
  
      title = file_obj.readline()
  
      def items(f, start_line_no, end_line_no):
          for line_no, line in enumerate(f, start=start_line_no+1):
              line = line.strip("\n").strip()
              if line.startswith(ignore_comments):
                  continue
              else:
                  for pattern, sub_str in regexp_subs:
                      line = re.sub(pattern, sub_str, line)
                  line = line.replace(chr(26), "")
                  if len(line) == 0:
                      continue
  
                  # for item in split_on_whitespace(line, sow_regex):
                  # for item in ["".join(t) for t in sow_regex.findall(line)]:
                  for item in ["".join(t) for t in line_splitter(line)]:
                      try:
                          yield np.float64(item)
                      except ValueError:
                          yield item
                  if line_no == end_line_no:
                      break
  
      logger.debug("Reading complete data section...")
      array = np.array(
          [i for i in items(file_obj, start_line_no=line_nos[0], end_line_no=line_nos[1])]
      )
      for value in value_null_subs:
          array[array == value] = np.nan
  
      logger.debug("Read {} items in data section".format(len(array)))
  
      # Cater for situations where the data section is empty.
      if len(array) == 0:
          logger.warning("Data section is empty therefore setting n_columns to zero")
          n_columns = 0
  
      # Re-shape the 1D array to a 2D array.
      if n_columns > 0:
          logger.debug("Attempt re-shape to {} columns".format(n_columns))
          try:
              array = np.reshape(array, (-1, n_columns))
          except ValueError as exception:
              error_message = "Cannot reshape ~A data size {0} into {1} columns".format(
                  array.shape, n_columns
              )
              if sys.version_info.major < 3:
                  exception.message = error_message
                  raise exception
              else:
                  raise ValueError(error_message).with_traceback(exception.__traceback__)
  
      # Identify how many columns have actually been found.
      if len(array.shape) < 2:
          arr_n_cols = 0
      else:
          arr_n_cols = array.shape[1]
  
      # Identify what the appropriate data types should be for each column based on the first
      # row of the data.
      if dtypes == "auto":
          if len(array) > 0:
              dtypes = identify_dtypes_from_data(array[0, :])
          else:
              dtypes = []
      elif dtypes is False:
          dtypes = [str for n in range(arr_n_cols)]
  
      # Iterate over each column, convert to the appropriate dtype (if possible)
      # and then yield the data column.
      for col_idx in range(arr_n_cols):
          curve_arr = array[:, col_idx]
          curve_dtype = dtypes[col_idx]
          try:
              curve_arr = curve_arr.astype(curve_dtype, copy=False)
          except ValueError:
              logger.warning(
                  "Could not convert curve #{} to {}".format(col_idx, curve_dtype)
              )
          else:
              logger.debug(
                  "Converted curve {} to {} ({})".format(col_idx, curve_dtype, curve_arr)
              )
          yield curve_arr
  
  
  def identify_dtypes_from_data(row):
      """Identify which columns should be 'str' and which 'float'.
  
      Args:
          row (1D ndarray): first row of data section
  
      Returns: list of [float, float, str, ...] etc
  
      """
      logger.debug("Creating auto dtype spec from first line of data array")
      dtypes_list = []
      for i, value in enumerate(row):
          try:
              value_converted = float(value)
          except:
              dtypes_list.append(str)
          else:
              dtypes_list.append(float)
          logger.debug(
              "Column {}: value {} -> dtype {}".format(i, value, dtypes_list[-1])
          )
      return dtypes_list
  
  
  def read_data_section_iterative_numpy_engine(file_obj, line_nos):
      """Read data section into memory.
  
      Arguments:
          file_obj: file-like object open for reading at the beginning of the section
          line_nos (tuple): the first and last line no of the section to read
  
  
      Returns:
          A numpy ndarray.
      """
  
      first_line = line_nos[0] + 1
      last_line = line_nos[1]
      max_rows = last_line - first_line
  
      file_obj.seek(0)
  
      # unpack=True transforms the data from an array of rows to an array of columns.
      # loose=False will throw an error on non-numerical data, which then sends the 
      # parsing to the 'normal' parser.
      array = np.genfromtxt(
          file_obj, skip_header=first_line, max_rows=max_rows, names=None, unpack=True, loose=False
      )
  
      # If there is only one data row, np.genfromtxt treats it as one array of
      # individual values. Lasio needs a array of arrays. This if statement
      # converts the single line data array to an array of arrays(column data).
      if len(array.shape) == 1:
          arr_len = array.shape[0]
          array = array.reshape(arr_len,1)
  
      return array
  
  
  def get_substitutions(read_policy, null_policy):
      """Parse read and null policy definitions into a list of regexp and value
      substitutions.
  
      Arguments:
          read_policy (str, list, or substitution): either (1) a string defined in
              defaults.READ_POLICIES; (2) a list of substitutions as defined by
              the keys of defaults.READ_SUBS; or (3) a list of actual substitutions
              similar to the values of defaults.READ_SUBS. You can mix (2) and (3)
              together if you want.
          null_policy (str, list, or sub): as for read_policy but for
              defaults.NULL_POLICIES and defaults.NULL_SUBS
  
      Returns:
          regexp_subs, value_null_subs, version_NULL - two lists and a bool.
          The first list is pairs of regexp patterns and substrs, and the second
          list is just a list of floats or integers. The bool is whether or not
          'NULL' was located as a substitution.
  
      """
      regexp_subs = []
      numerical_subs = []
      version_NULL = False
  
      for policy_typ, policy, policy_subs, subs in (
          ("read", read_policy, defaults.READ_POLICIES, defaults.READ_SUBS),
          ("null", null_policy, defaults.NULL_POLICIES, defaults.NULL_SUBS),
      ):
          try:
              is_policy = policy in policy_subs
          except TypeError:
              is_policy = False
          if is_policy:
              logger.debug('using {} policy of "{}"'.format(policy_typ, policy))
              all_subs = []
              for sub in policy_subs[policy]:
                  logger.debug("adding substitution {}".format(sub))
                  if sub in subs:
                      all_subs += subs[sub]
                  if sub == "NULL":
                      logger.debug("located substitution for LAS.version.NULL as True")
                      version_NULL = True
          else:
              all_subs = []
              for item in policy:
                  if item in subs:
                      all_subs += subs[item]
                      if item == "NULL":
                          logger.debug(
                              "located substitution for LAS.version.NULL as True"
                          )
                          version_NULL = True
                  else:
                      all_subs.append(item)
          for item in all_subs:
              try:
                  iter(item)
              except TypeError:
                  logger.debug("added numerical substitution: {}".format(item))
                  numerical_subs.append(item)
              else:
                  logger.debug(
                      'added regexp substitution: pattern={} substr="{}"'.format(
                          item[0], item[1]
                      )
                  )
                  regexp_subs.append(item)
      numerical_subs = [n for n in numerical_subs if not n is None]
  
      return regexp_subs, numerical_subs, version_NULL
  
  
  def parse_header_items
  
  [... the rest is omitted ...]
files: lasio/reader.py