Select specific columns in a text file by their column names and extract their contents
Question: I am a beginner in Python and I am finding it very difficult to come up with
the correct solution for this problem. I glanced through all the similar posts
in stackoverflow and couldn't find the solution.  
I have a ".ext" file. I need to skip first two lines. The third line has the
column names for the table.  
I need to search for the columns omega(n,n) and Sigma(n,n) column names where
n can be any number (Eg:sigma(1,1), omega(2,2)). Analyse the columns with
column names "sigma(n,n)" and "omega(n,n)" and check the values of these
columns for the row starting with '-1000000000'.If the value is <0.001, output
"true".

my code is:

    
    
    import numpy as np
    array=[]
    array1=[]
    b = np.genfromtxt(r'C:/nm73/proj/one.ext', delimiter=' ', names=True,dtype=None)[3:,:]
    for n in range(len(b)-1):
        array=b['Sigma(n,n)']
        array1=b['omega(n,n)']
    

I don't know how to check the elements.

One.ext file is as shown below: I apologize if the file in not in correct
format. I am new to stackoverflow. Any help is highly appreciated.

    
    
    TABLE NO.     1: First Order Conditional Estimation with Interaction: Goal     Function=MINIMUM VALUE OF OBJECTIVE FUNCTION: Problem=1 Subproblem=0 Superproblem1=0     Iteration1=0 Superproblem2=0 Iteration2=0
     ITERATION    THETA1       THETA2       SIGMA(1,1)   SIGMA(2,1)   SIGMA(2,2)   OMEGA(1,1)   OMEGA(2,1)   OMEGA(2,2)   OBJ
                0  2.50000E-01  1.00000E+01  1.00000E-01  0.00000E+00  1.00000E-01      1.00000E-01  0.00000E+00  1.00000E-01    9436.65314342255
                5  2.34948E-01  3.67675E+00  9.04159E-02  0.00000E+00  2.74933E+00  1.98686E-01  0.00000E+00  1.75724E-01    8745.97204613658
               10  2.11090E-01  4.30565E+00  1.34312E-01  0.00000E+00  1.12619E+00  1.32484E-01  0.00000E+00  1.36824E-02    8595.43106384756
               15  2.10696E-01  4.35495E+00  1.23897E-01  0.00000E+00  1.29124E+00  1.28600E-01  0.00000E+00  1.24441E-02    8591.51400321872
               20  2.11129E-01  4.36325E+00  1.24283E-01  0.00000E+00  1.28733E+00  1.28815E-01  0.00000E+00  1.24211E-02    8591.50022332770
      -1000000000  2.11129E-01  4.36325E+00  1.24283E-01  0.00000E+00  1.28733E+00  1.28815E-01  0.00000E+00  1.24211E-02    8591.50022332770
      -1000000001  8.07565E-03  6.97861E-02  5.28558E-03  1.00000E+10  4.20370E-01  1.78706E-02  1.00000E+10  3.15324E-03   0.000000000000000E+000
      -1000000004  0.00000E+00  0.00000E+00  3.52538E-01  0.00000E+00  1.13460E+00  3.58908E-01  0.00000E+00  1.11450E-01   0.000000000000000E+000
      -1000000005  0.00000E+00  0.00000E+00  7.49648E-03  1.00000E+10  1.85250E-01  2.48957E-02  1.00000E+10  1.41465E-02   0.000000000000000E+000
    


Answer: If you don't specify `delimiter`, then all _consecutive_ whitespace will be
understood to act as one delimiter. If you specify `delimiter=' '` then
literally _each_ space will act as a delimiter. That leads to a ValueError,
since `genfromtxt` will expect the wrong number of columns.

So if instead you use:

    
    
    In [396]: b = np.genfromtxt(filename, names=True, dtype=None, skip_header=1)
    

Then you'll end up with a structured array like this:

    
    
    In [397]: b
    Out[397]: 
    array([(0, 0.25, 10.0, 0.1, 0.0, 0.1, 0.1, 0.0, 0.1, 9436.65314342255),
           (5, 0.234948, 3.67675, 0.0904159, 0.0, 2.74933, 0.198686, 0.0, 0.175724, 8745.97204613658),
           (10, 0.21109, 4.30565, 0.134312, 0.0, 1.12619, 0.132484, 0.0, 0.0136824, 8595.43106384756),
           (15, 0.210696, 4.35495, 0.123897, 0.0, 1.29124, 0.1286, 0.0, 0.0124441, 8591.51400321872),
           (20, 0.211129, 4.36325, 0.124283, 0.0, 1.28733, 0.128815, 0.0, 0.0124211, 8591.5002233277),
           (-1000000000, 0.211129, 4.36325, 0.124283, 0.0, 1.28733, 0.128815, 0.0, 0.0124211, 8591.5002233277),
           (-1000000001, 0.00807565, 0.0697861, 0.00528558, 10000000000.0, 0.42037, 0.0178706, 10000000000.0, 0.00315324, 0.0),
           (-1000000004, 0.0, 0.0, 0.352538, 0.0, 1.1346, 0.358908, 0.0, 0.11145, 0.0),
           (-1000000005, 0.0, 0.0, 0.00749648, 10000000000.0, 0.18525, 0.0248957, 10000000000.0, 0.0141465, 0.0)], 
          dtype=[('ITERATION', '<i4'), ('THETA1', '<f8'), ('THETA2', '<f8'), ('SIGMA11', '<f8'), ('SIGMA21', '<f8'), ('SIGMA22', '<f8'), ('OMEGA11', '<f8'), ('OMEGA21', '<f8'), ('OMEGA22', '<f8'), ('OBJ', '<f8')])
    

Notice the `dtype` at the end. The column names do not contain parentheses or
commas, so instead of `SIGMA(1,1)` you have `SIGMA11`. You can access this
column like this:

    
    
    In [398]: b['SIGMA11']
    Out[398]: 
    array([ 0.1       ,  0.0904159 ,  0.134312  ,  0.123897  ,  0.124283  ,
            0.124283  ,  0.00528558,  0.352538  ,  0.00749648])
    

