Crawling SPOJ through cURL and C++ I am trying to write industry standard code.
https://www.quora.com/How-do-I-follow-a-user-on-Spoj-for-solving-problems-Refer-Details

Someone gave me this A2A.
And I wrote this code for it

//compile with g++ -std=c++11
#include &lt;bits/stdc++.h&gt;
using namespace std;
struct PROBLEM
{
    string name;
    string links;
    int users;
    double points;
};
bool compare (PROBLEM &amp; p1 , PROBLEM &amp; p2)
{
    return p1.users &lt; p2.users;
}
void savePage(string &amp; username)
{
    string url = "http://www.spoj.com/users/" + username +"/";
    string cmd = "curl " + url + " &gt;" + username + ".txt"; 
    system (cmd.c_str());
}
bool checkRegex(string &amp; str)
{
    return regex_match (str , std::regex("^\\s{0,}&lt;td\\salign.{0,}$"));
}
bool checkRegexProb(string &amp; str)
{
    return regex_match(str , std::regex("^\\s{0,}&lt;td\\sclass=\"t.{0,}$"));
}
int getNum(string &amp; str)
{
    int i;
    string temp;
    for (i = 0 ; str[i] != '&gt;' ; i++);
    i++;    
    for(;str[i] != '&lt;' ; i++)
        temp += str[i];
    return stoi(temp);
}
int getUsers(string &amp; str)
{
    string url = "http://www.spoj.com/ranks/" + str + "/";
    string cmd = "curl " +url + " &gt;" + str +".txt";
    system(cmd.c_str());
    string filename = str + ".txt";
    int Count = 0;
    ifstream myfile(filename.c_str());
    if (myfile)  
    {
        string line;
        while (getline( myfile, line ))  
        {
            if (checkRegexProb(line) )
            {
                Count ++;
                if(Count == 2)
                {
                    myfile.close();
                    string deleteFile = "rm -r " + filename;
                    system(deleteFile.c_str());  
                    return getNum(line);
                }           
            }   
        }
    }   
}
vector&lt;PROBLEM&gt; readFile(string &amp; username)
{
    std::vector&lt;PROBLEM&gt; v;
    string filename = username + ".txt";
    ifstream myfile(filename.c_str());
    if (myfile)  
    {
        string line;
        while (getline( myfile, line ))  
        {
            if (checkRegex(line) )
            {
                string temp;
                size_t found = line.find("/status/");
                for(size_t i = found + 8 ; line[i]!=',' ; i++)
                    temp += line[i];
                if(temp != "")
                {
                    PROBLEM p;
                    p.name = temp;
                    p.users = getUsers(p.name);
                    while(p.users == 0)
                        p.users = getUsers(p.name); // bug -&gt; curl gives empty response
                    v.push_back(p);
                }   
            }
        }
        myfile.close();
    }
    return v;

}
void makeLink(std::vector&lt;PROBLEM&gt; &amp; v)
{
    for(int i = 0 ; i &lt; v.size() ; i++)
        v[i].points = 80.0/(40.0+ v[i].users),
        v[i].links = "&lt;a href =\" http:://www.spoj.com/problems/" + v[i].name +"\"&gt; " + v[i].name +" "+ to_string(v[i].points) +"&lt;/a&gt; &lt;br&gt;";
}
string makeHtml(std::vector&lt;PROBLEM&gt; &amp; v)
{
    string list ;
    for (int i = 0 ; i &lt; v.size() ; i++ )
        list += v[i].links;
    string HTML = 
    "&lt;html&gt;&lt;body&gt;"+ list +"&lt;/body&gt;&lt;/html&gt;";
    return HTML;
}
int main(int argc, char const *argv[])
{
    string username ;
    cin &gt;&gt; username ;
    savePage(username) ;
    std::vector&lt;PROBLEM&gt; v = readFile(username);
    sort(v.rbegin(), v.rend(),compare);
    makeLink(v);
    std::ofstream outfile ((username + ".html").c_str());
    outfile &lt;&lt;makeHtml(v);
    system(("rm -f "+ username + ".txt").c_str());
    system(("gnome-open " + username + ".html").c_str());   
    return 0;
}


How can I improve it so that it matches industry standard?
A few more possibilities to consider:

pre-compille your regular expressions

Compiling a regular expression is (potentially) a somewhat expensive operation. In this case, the regular expressions remain constant throughout the program, so you might as well compile them once, and use the compiled regular expression after that:

bool checkRegex(string &amp; str)
{
    static std::regex pattern{"^\\s{0,}&lt;td\\salign.{0,}$"};

    return regex_match (str , pattern);
}


use operator overloading where sensible

For example, your compare function:

bool compare (PROBLEM &amp; p1 , PROBLEM &amp; p2)
{
    return p1.users &lt; p2.users;
}


...really makes more sense as an overload of operator&lt;:

bool operator&lt;(PROBLEM &amp; p1 , PROBLEM &amp; p2)
{
    return p1.users &lt; p2.users;
}


const correctness

Taking the operator above a step further, we don't want x &lt; y to modify either x or y, so we should pass the operands by const reference:

bool operator&lt;(PROBLEM const &amp;p1 , PROBLEM const &amp;p2)
{
    return p1.users &lt; p2.users;
}


This way, when we sort our vector of Problem objects, we don't need to explicitly specify how to compare them:

sort(v.rbegin(), v.rend());


Use the force library, Luke

In addition to a command line utility, cURL is available as a library that you can use in your program. There's also a cURLpp library that provides a C++ wrapper around libcurl, which you may prefer. Either can help you write code that's quite a bit simpler and more readable.

Work with the library, not against it

Right now, you have (for example) a readFile to read the contents of a file into a vector of PROBLEM. That's not really how the standard library is meant to work though. To work optimally with it, you really just want to overload operator&gt;&gt; to read one PROBLEM from the file. Once you have that, you can (for example) use stream iterators to read from a file into a vector.

class Problem { 
    // ...

    friend std::istream &amp;operator&gt;&gt;(std::istream &amp;is, Problem &amp;p) { 
        // code to read one Problem from the stream goes here
        return is;
    }
};


Then we can initialize our vector from a pair of iterators:

std::ifstream in("input.txt");
std::vector&lt;Problem&gt; v{ std::istream_iterator&lt;Problem&gt;(in), {} };