Method to remove comments from a string line There is some language where the following are considered comments:


[str] // comment
[str] /* comment */ [str]
[str] # comment
[str] /* comment1
comment2 */ [str]


where [str] is 0 or more occurrences of some string.

I'm reading a file line by line, and need to remove the comments from the whole file (the file is large so I cannot keep the whole content in memory).

I have written a method that receives each line, removes the comments from it, and returns the remaining string.

(Here s_bInComment is a static variable in Parser class with initial value of false)

void Parser::removeComments(std::string &amp;str)
{
    // see if we're still in comment lines, i.e. not reached */
    auto commentEnd = str.find("*/");
    if (s_bInComment &amp;&amp; commentEnd == std::string::npos)
    {
        str.clear();
        return;
    }

    std::string result;

    const auto iStrLen = str.length();
    for (std::size_t idx = 0; idx &lt; iStrLen; ++idx)
    {
        if (str[idx] == '/')
        {
            if (idx + 1 &lt; iStrLen)
            {
                if (str[idx + 1] == '/')
                {
                    // we found //
                    // take the part of the string before the comment, if any
                    str = str.substr(0, idx);
                    break;
                }

                if (str[idx + 1] == '*')
                {
                    // see if */ is in the same line
                    commentEnd = str.find("*/");
                    if (commentEnd == std::string::npos)
                    {
                        // multiline comment
                        s_bInComment = true;
                        str = str.substr(0, idx);
                    }
                    else
                    {
                        // remove the comment
                        str = result + str.substr(commentEnd + 2);

                        // recurse on the remaining string
                        removeComments(str);
                    }

                    break;
                }
            }
        }
        else if (str[idx] == '*')
        {
            if (idx + 1 &lt; iStrLen)
            {
                if (str[idx + 1] == '/')
                {
                    // we have found */, comment ended
                    s_bInComment = false;

                    // take the part of the string after the comment, if any
                    str = str.substr(idx + 2);

                    // recurse on the remaining string
                    removeComments(str);

                    break;
                }
            }
        }
        else if (str[idx] == '#')
        {
            // if there's a text before #, it should be accumulated in result
            str = result.size() ? result : std::string();
            break;
        }
        else if (!s_bInComment)
            result += str[idx];
    }
}


I have tested my code with the following input lines:


a /* b */ c
a /* b */
a // b
a /* b
c
*/
// a
a /* b
*/ c
# a
a # b
a /* a */ b /* b */ c


Please let me know if there's room for improvement in one or other part.
I believe it's a good example of a problem that you can represent with a state-machine. What you do with the next character depends on the state you're in: 


if you're outside of a comment, then you copy the character unless it's a / or a #
if the last character was a / outside a comment, then test if the new character is a * or a /
if you're inside a single line comment, you ignore it unless it's a newline
if you're inside a multi-line comment, you ignore it unless it's a star
if you've just read a * inside a multi-line comment then you need to check if the new char isn't a /


State-machines aren't always the most elegant way of coding, but they're robust, easy to read and blazingly fast. So how would you implement it in this case?

First, don't read the input line by line. It is probably a mistake since crucial information resides in the previous line, if there's one: you can't know, looking at a line, if a multi-line comment was opened in the previous one. Also, it will force you to perform memory management each time you read a line (you need to check if the output string is big enough and allocate more memory if it isn't), which will slow down your program, whereas looking character by character will allow you greater flexibility (any object exposing iterators on characters is acceptable) and prevent allocations if they aren't needed.

That brings us to the signature:

template &lt;typename InputIt, typename OutputIt&gt;
OutputIt copy_without_comments(InputIt first, InputIt last, OutputIt out)


The OutputIt can be many things: an iterator on an allocated buffer, an iterator constructing a string, an iterator on a stream...

The state machine itself is generally implemented with an enum specifying the possible states:

enum class State : char { SlashOutsideComment, StarInsideComment, SingleLineComment, MultiLineComment, NotAComment };


Then it's only a big switch to know how to handle the next character:

 switch (state) {
            case State::SlashOutsideComment:
                if (*first == '/') state = State::SingleLineComment;
                else if (*first == '*') state = State::MultiLineComment;
                else {
                    state = State::NotAComment;
                    *out++ = '/';
                    *out++ = *first;
                }
                break;
            case State::StarInsideComment:
                if (*first == '/') state = State::NotAComment;
                else state = State::MultiLineComment;
                break;
            case State::NotAComment:
                if (*first == '#') state = State::SingleLineComment;
                else if (*first == '/') state = State::SlashOutsidedComment;
                else *out++ = *first;
                break;
            case State::SingleLineComment:
                if (*first == '\n') {
                    state = State::NotAComment;
                    *out++ = '\n';
                }
                break;
            case State::MultiLineComment:
                if (*first == '*') state = State::StarInsideComment;
        }


Link to test and play with: https://wandbox.org/permlink/iXC7DWaU8Tk8jrf3