Optimizating my code simulating a database (2)
<p>Some days ago I made you a question and I got some really useful answers. I will make a summary to those of you who didn't read and I will explain my new doubts and where I have problems now.</p>

<p><strong>Explanation</strong></p>

<p>I have been working on a program, simulating a small database, that first of all read information from txt files and store them in the computer memory and then, I can make queries taking normal tables and/or transposed tables. The problem is that the performance is not good enough yet. It works slower than what I expect. I have improved it but I think I should improve it more. I have specific points where my program doesn't have a good performance.</p>

<p><strong>Current problem</strong></p>

<p>The first problem that I have now (where my program is slower) is that I spend more time to, for example table with 100,000 columns &amp; 100 rows (0.325 min, I've improved this thanks to your help) than 100,000 rows &amp; 100 columns (1.61198 min, the same than before). But on the other hand, access time to some data is better in the second case (in a determined example, 47 seconds vs. 6079 seconds in the first case) any idea why??</p>

<p><strong>Explanation</strong></p>

<p>Now let me remind you how my code works (with an atached summary of my code)</p>

<p>First of all I have a .txt file simulating a database table with random strings separated with "|". Here you have an example of table (with 7 rows and 5 columns). I also have the transposed table</p>

<p><em><strong>NormalTable.txt</em></strong></p>

<pre><code>42sKuG^uM|24465\lHXP|2996fQo\kN|293cvByiV|14772cjZ`SN|
28704HxDYjzC|6869xXj\nIe|27530EymcTU|9041ByZM]I|24371fZKbNk|
24085cLKeIW|16945TuuU\Nc|16542M[Uz\|13978qMdbyF|6271ait^h|
13291_rBZS|4032aFqa|13967r^\\`T|27754k]dOTdh|24947]v_uzg|
1656nn_FQf|4042OAegZq|24022nIGz|4735Syi]\|18128klBfynQ|
6618t\SjC|20601S\EEp|11009FqZN|20486rYVPR|7449SqGC|
14799yNvcl|23623MTetGw|6192n]YU\Qe|20329QzNZO_|23845byiP|
</code></pre>

<p><em><strong>TransposedTable.txt</em></strong> (This is new from the previous post)</p>

<pre><code>42sKuG^uM|28704HxDYjzC|24085cLKeIW|13291_rBZS|1656nn_FQf|6618t\SjC|14799yNvcl|
24465\lHXP|6869xXj\nIe|16945TuuU\Nc|4032aFqa|4042OAegZq|20601S\EEp|23623MTetGw|
2996fQo\kN|27530EymcTU|16542M[Uz\|13967r^\\`T|24022nIGz|11009FqZN|6192n]YU\Qe|
293cvByiV|9041ByZM]I|13978qMdbyF|27754k]dOTdh|4735Syi]\|20486rYVPR|20329QzNZO_|
14772cjZ`SN|24371fZKbNk|6271ait^h|24947]v_uzg|18128klBfynQ|7449SqGC|23845byiP|
</code></pre>

<p><strong>Explanation</strong></p>

<p>This information in a .txt file is read by my program and stored in the computer memory. Then, when making queries, I will access to this information stored in the computer memory. Loading the data in the computer memory can be a slow process, but accessing to the data later will be faster, what really matters me. </p>

<p>Here you have the part of the code that read this information from a file and store in the computer.</p>

<p><em><strong>Code that reads data from the Table.txt file and store it in the computer memory</em></strong></p>

<pre><code>int h;
do
{
    cout&lt;&lt; "Do you want to query the normal table or the transposed table? (1- Normal table/ 2- Transposed table):" ;
    cin&gt;&gt;h; 
}while(h!=1 &amp;&amp; h!=2);

string ruta_base("C:\\Users\\Raul Velez\\Desktop\\Tables\\");
if(h==1)
{
    ruta_base +="NormalTable.txt"; // Folder where my "Table.txt" is found
}

if(h==2)
{
    ruta_base +="TransposedTable.txt";
}

string temp; // Variable where every row from the Table.txt file will be firstly stored
vector&lt;string&gt; buffer; // Variable where every different row will be stored after separating the different elements by tokens.
vector&lt;ElementSet&gt; RowsCols; // Variable with a class that I have created, that simulated a vector and every vector element is a row of my table

ifstream ifs(ruta_base.c_str());
while(getline( ifs, temp )) // We will read and store line per line until the end of the ".txt" file. 
{
    size_t tokenPosition = temp.find("|"); // When we find the simbol "|" we will identify different element. So we separate the string temp into tokens that will be stored in vector&lt;string&gt; buffer
    // --- NEW PART ------------------------------------
    const char* p = temp.c_str();
    char* p1 = strdup(p);

    char* pch = strtok(p1, "|");
    while(pch)
    {
            buffer.push_back(string(pch));
            pch = strtok(NULL,"|");
    }
    free(p1);

    ElementSet sss(0,buffer);
    buffer.clear();
    RowsCols.push_back(sss); // We store all the elements of every row (stores as vector&lt;string&gt; buffer) in a different position in "RowsCols" 
    // --- NEW PART END ------------------------------------
}

Table TablesStorage(RowsCols); // After every loop we will store the information about every .txt file in the vector&lt;Table&gt; TablesDescriptor
vector&lt;Table&gt; TablesDescriptor;
TablesDescriptor.push_back(TablesStorage); // In the vector&lt;Table&gt; TablesDescriptor will be stores all the different tables with all its information

DataBase database(1, TablesDescriptor);
</code></pre>

<p><strong>Information already given in the previous post</strong></p>

<p>After this, comes the access to the information part. Let's suppose that I want to make a query, and I ask for input. Let's say that my query is row "n", and also the consecutive tuples "numTuples", and the columns "y". (We must say that the number of columns is defined by a decimal number "y", that will be transformed into binary and will show us the columns to be queried, for example, if I ask for columns 54 (00110110 in binary) I will ask for columns 2, 3, 5 and 6). Then I access to the computer memory to the required information and store it in a vector shownVector. Here I show you the part of this code.</p>

<p><strong>Problem</strong>
In the loop <em>if(h == 2)</em> where data from the transposed tables are accessed, performance is poorer ¿why?</p>

<p><em><strong>Code that access to the required information upon my input</em></strong></p>

<pre><code>int n, numTuples; 
unsigned long long int y;

cout&lt;&lt; "Write the ID of the row you want to get more information: " ;
cin&gt;&gt;n; // We get the row to be represented -&gt; "n"

cout&lt;&lt; "Write the number of followed tuples to be queried: " ;
cin&gt;&gt;numTuples; // We get the number of followed tuples to be queried-&gt; "numTuples"

cout&lt;&lt;"Write the ID of the 'columns' you want to get more information: ";
cin&gt;&gt;y; // We get the "columns" to be represented ' "y"

unsigned int r; // Auxiliar variable for the columns path
int t=0; // Auxiliar variable for the tuples path
int idTable;

vector&lt;int&gt; columnsToBeQueried; // Here we will store the columns to be queried get from the bitset&lt;500&gt; binarynumber, after comparing with a mask
vector&lt;string&gt; shownVector; // Vector to store the final information from the query
bitset&lt;5000&gt; mask;
mask=0x1;

clock_t t1, t2;
t1=clock(); // Start of the query time

bitset&lt;5000&gt; binaryNumber = Utilities().getDecToBin(y); // We get the columns -&gt; change number from decimal to binary. Max number of columns: 5000

// We see which columns will be queried
for(r=0;r&lt;binaryNumber.size();r++) //
{               
    if(binaryNumber.test(r) &amp; mask.test(r))  // if both of them are bit "1"
    {
        columnsToBeQueried.push_back(r);
    }
    mask=mask&lt;&lt;1;   
}

do
{
    for(int z=0;z&lt;columnsToBeQueried.size();z++)
    {
        ElementSet selectedElementSet;
        int i;
        i=columnsToBeQueried.at(z);
        Table&amp; selectedTable = database.getPointer().at(0); // It simmulates a vector with pointers to different tables that compose the database, but our example database only have one table, so don't worry ElementSet selectedElementSet;
        if(h == 1)
        {

            selectedElementSet=selectedTable.getRowsCols().at(n);
            shownVector.push_back(selectedElementSet.getElements().at(i)); // We save in the vector shownVector the element "i" of the row "n"
        }

        if(h == 2)  
        {
            selectedElementSet=selectedTable.getRowsCols().at(i);
            shownVector.push_back(selectedElementSet.getElements().at(n)); // We save in the vector shownVector the element "n" of the row "i"
        }
        n=n+1;
        t++;            
    }
}while(t&lt;numTuples);

t2=clock(); // End of the query time
showVector().finalVector(shownVector);
float diff ((float)t2-(float)t1);
float microseconds = diff / CLOCKS_PER_SEC*1000000;
cout&lt;&lt;"Time: "&lt;&lt;microseconds&lt;&lt;endl;
</code></pre>

<p><em><strong>Class definitions</strong></em></p>

<p>Here I attached some of the class definitions so that you can compile the code, and understand better how it works:</p>

<pre><code>class ElementSet
{
private:
    int id;
    vector&lt;string&gt; elements; 

public:
    ElementSet(); 
    ElementSet(int, vector&lt;string&gt;&amp;); 

    const int&amp; getId();
    void setId(int);

    const vector&lt;string&gt;&amp; getElements();
    void setElements(vector&lt;string&gt;);

};

class Table
{
private:
    vector&lt;ElementSet&gt; RowsCols; 

public:
    Table(); 
    Table(vector&lt;ElementSet&gt;&amp;); 

    const vector&lt;ElementSet&gt;&amp; getRowsCols();
    void setRowsCols(vector&lt;ElementSet&gt;);
};


class DataBase
{
     private:
        int id;
        vector&lt;Table&gt; pointer; 

     public:
        DataBase(); 
        DataBase(int, vector&lt;Table&gt;&amp;); 

    const int&amp; getId();
    void setId(int);

    const vector&lt;Table&gt;&amp; getPointer();
    void setPointer(vector&lt;Table&gt;);

    };

class Utilities
{
        public:
        Utilities();
        static bitset&lt;500&gt; getDecToBin(unsigned long long int);
};
</code></pre>

<p><strong>Summary of my problems</strong></p>

<ul>
<li>Why the load of the data is different depending on the table format???</li>
<li>Why the access to the information also depends on the table (and the performance is in the opposite way than the table data load?</li>
</ul>

<p>Thank you very much for all your help!!! :)</p>
