title: Weird SIGSEGV segmentation fault in std::string::assign() method from libstdc++.so.6
body: <p>My program recently encountered a weird segfault when running.  <strong>I want to know if somebody had met this error before and how it could be fixed.</strong>  Here is more info:</p>
  <p><strong>Basic info:</strong></p>
  <ul>
  <li>CentOS 5.2, kernal version is 2.6.18</li>
  <li>g++ (GCC) 4.1.2 20080704 (Red Hat 4.1.2-50)</li>
  <li>CPU: Intel x86 family</li>
  <li>libstdc++.so.6.0.8</li>
  <li>My program will start multiple threads to process data.  The segfault occurred in one of the threads.</li>
  <li>Though it's a multi-thread program, the segfault seemed to occur on a local std::string object.  I'll show this in the code snippet later.</li>
  <li>The program is compiled with -g, -Wall and -fPIC, and without -O2 or other optimization options.</li>
  </ul>
  <p><strong>The core dump info:</strong></p>
  <pre><code>Core was generated by `./myprog'.
  Program terminated with signal 11, Segmentation fault.
  #0  0x06f6d919 in __gnu_cxx::__exchange_and_add(int volatile*, int) () from /usr/lib/libstdc++.so.6
  (gdb) bt
  #0  0x06f6d919 in __gnu_cxx::__exchange_and_add(int volatile*, int) () from /usr/lib/libstdc++.so.6
  #1  0x06f507c3 in std::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::assign(std::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;) () from /usr/lib/libstdc++.so.6
  #2  0x06f50834 in std::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::operator=(std::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;) () from /usr/lib/libstdc++.so.6
  #3  0x081402fc in Q_gdw::ProcessData (this=0xb2f79f60) at ../../../myprog/src/Q_gdw/Q_gdw.cpp:798
  #4  0x08117d3a in DataParser::Parse (this=0x8222720) at ../../../myprog/src/DataParser.cpp:367
  #5  0x08119160 in DataParser::run (this=0x8222720) at ../../../myprog/src/DataParser.cpp:338
  #6  0x080852ed in Utility::__dispatch (arg=0x8222720) at ../../../common/thread/Thread.cpp:603
  #7  0x0052c832 in start_thread () from /lib/libpthread.so.0
  #8  0x00ca845e in clone () from /lib/libc.so.6
  </code></pre>
  <p>Please note that the segfault begins within the <strong>basic_string::operator=()</strong>.</p>
  <p><strong>The related code:</strong>
  (I've shown more code than that might be needed, and please ignore the coding style things for now.)</p>
  <pre><code>int Q_gdw::ProcessData()
  {
      char tmpTime[10+1] = {0};
      char A01Time[12+1] = {0};
      std::string tmpTimeStamp;
  
      // Get the timestamp from TP
      if((m_BackFrameBuff[11] &amp; 0x80) &gt;&gt; 7)
      {
          for (i = 0; i &lt; 12; i++)
          {
              A01Time[i] = (char)A15Result[i];
          }
          tmpTimeStamp = FormatTimeStamp(A01Time, 12);  // Segfault occurs on this line
  </code></pre>
  <p>And here is the prototype of this FormatTimeStamp method:</p>
  <pre><code>std::string FormatTimeStamp(const char *time, int len)
  </code></pre>
  <p>I think such string assignment operations should be a kind of commonly used one, but I just don't understand why a segfault could occurr here.</p>
  <p><strong>What I have investigated:</strong></p>
  <p>I've searched on the web for answers.  I looked at <a href="http://gcc.gnu.org/ml/libstdc++/2007-12/msg00035.html" rel="nofollow noreferrer">here</a>.  The reply says try to recompile the program with _GLIBCXX_FULLY_DYNAMIC_STRING macro defined.  I tried but the crash still happens.</p>
  <p>I also looked at <a href="http://cygwin.com/ml/cygwin/2005-10/msg00051.html" rel="nofollow noreferrer">here</a>.  It also says to recompile the program with _GLIBCXX_FULLY_DYNAMIC_STRING, but the author seems to be dealing with a different problem with mine, thus I don't think his solution works for me.</p>
  <hr />
  <p><strong>Updated on 08/15/2011</strong></p>
  <p>Here is the original code of this FormatTimeStamp.  I understand the coding doesn't look very nice(too many magic numbers, for instance..), but let's focus on the crash issue first.</p>
  <pre><code>string Q_gdw::FormatTimeStamp(const char *time, int len)
  {
      string timeStamp;
      string tmpstring;
  
      if (time)  // It is guaranteed that &quot;time&quot; is correctly zero-terminated, so don't worry about any overflow here.
          tmpstring = time;
  
      // Get the current time point.
      int year, month, day, hour, minute, second;
  #ifndef _WIN32
      struct timeval timeVal;
      struct tm *p;
      gettimeofday(&amp;timeVal, NULL);
      p = localtime(&amp;(timeVal.tv_sec));
      year = p-&gt;tm_year + 1900;
      month = p-&gt;tm_mon + 1;
      day = p-&gt;tm_mday;
      hour = p-&gt;tm_hour;
      minute = p-&gt;tm_min;
      second = p-&gt;tm_sec;
  #else
      SYSTEMTIME sys;
      GetLocalTime(&amp;sys);
      year = sys.wYear;
      month = sys.wMonth;
      day = sys.wDay;
      hour = sys.wHour;
      minute = sys.wMinute;
      second = sys.wSecond;
  #endif
  
      if (0 == len)
      {
          // The &quot;time&quot; doesn't specify any time so we just use the current time
          char tmpTime[30];
          memset(tmpTime, 0, 30);
          sprintf(tmpTime, &quot;%d-%d-%d %d:%d:%d.000&quot;, year, month, day, hour, minute, second);
          timeStamp = tmpTime;
      }
      else if (6 == len)
      {
          // The &quot;time&quot; specifies &quot;day-month-year&quot; with each being 2-digit.
          // For example: &quot;150811&quot; means &quot;August 15th, 2011&quot;.
          timeStamp = &quot;20&quot;;
          timeStamp = timeStamp + tmpstring.substr(4, 2) + &quot;-&quot; + tmpstring.substr(2, 2) + &quot;-&quot; +
                  tmpstring.substr(0, 2);
      }
      else if (8 == len)
      {
          // The &quot;time&quot; specifies &quot;minute-hour-day-month&quot; with each being 2-digit.
          // For example: &quot;51151508&quot; means &quot;August 15th, 15:51&quot;.
          // As the year is not specified, the current year will be used.
          string strYear;
          stringstream sstream;
          sstream &lt;&lt; year;
          sstream &gt;&gt; strYear;
          sstream.clear();
  
          timeStamp = strYear + &quot;-&quot; + tmpstring.substr(6, 2) + &quot;-&quot; + tmpstring.substr(4, 2) + &quot; &quot; +
                  tmpstring.substr(2, 2) + &quot;:&quot; + tmpstring.substr(0, 2) + &quot;:00.000&quot;;
      }
      else if (10 == len)
      {
          // The &quot;time&quot; specifies &quot;minute-hour-day-month-year&quot; with each being 2-digit.
          // For example: &quot;5115150811&quot; means &quot;August 15th, 2011, 15:51&quot;.
          timeStamp = &quot;20&quot;;
          timeStamp = timeStamp + tmpstring.substr(8, 2) + &quot;-&quot; + tmpstring.substr(6, 2) + &quot;-&quot; + tmpstring.substr(4, 2) + &quot; &quot; +
                  tmpstring.substr(2, 2) + &quot;:&quot; + tmpstring.substr(0, 2) + &quot;:00.000&quot;;
      }
      else if (12 == len)
      {
          // The &quot;time&quot; specifies &quot;second-minute-hour-day-month-year&quot; with each being 2-digit.
          // For example: &quot;305115150811&quot; means &quot;August 15th, 2011, 15:51:30&quot;.
          timeStamp = &quot;20&quot;;
          timeStamp = timeStamp + tmpstring.substr(10, 2) + &quot;-&quot; + tmpstring.substr(8, 2) + &quot;-&quot; + tmpstring.substr(6, 2) + &quot; &quot; +
                  tmpstring.substr(4, 2) + &quot;:&quot; + tmpstring.substr(2, 2) + &quot;:&quot; + tmpstring.substr(0, 2) + &quot;.000&quot;;
      }
  
      return timeStamp;
  }
  </code></pre>
  <hr />
  <p><strong>Updated on 08/19/2011</strong></p>
  <p>This problem has finally been addressed and fixed.  The FormatTimeStamp() function has nothing to do with the root cause, in fact.  The segfault is caused by a writing overflow of a local char buffer.</p>
  <p>This problem can be reproduced with the following simpler program(please ignore the bad namings of some variables for now):</p>
  <p>(Compiled with &quot;g++ -Wall -g main.cpp&quot;)</p>
  <pre><code>#include &lt;string&gt;
  #include &lt;iostream&gt;
  
  void overflow_it(char * A15, char * A15Result)
  {
      int m;
      int t = 0,i = 0;
      char temp[3];
  
      for (m = 0; m &lt; 6; m++)
      {
          t = ((*A15 &amp; 0xf0) &gt;&gt; 4) *10 ;
          t += *A15 &amp; 0x0f;
          A15 ++;
          
          std::cout &lt;&lt; &quot;m = &quot; &lt;&lt; m &lt;&lt; &quot;; t = &quot; &lt;&lt; t &lt;&lt; &quot;; i = &quot; &lt;&lt; i &lt;&lt; std::endl;
          
          memset(temp, 0, sizeof(temp));
          sprintf((char *)temp, &quot;%02d&quot;, t);   // The buggy code: temp is not big enough when t is a 3-digit integer.
          A15Result[i++] = temp[0];
          A15Result[i++] = temp[1];
      }
  }
  
  int main(int argc, char * argv[])
  {
      std::string str;
  
      {
          char tpTime[6] = {0};
          char A15Result[12] = {0};
          
          // Initialize tpTime
          for(int i = 0; i &lt; 6; i++)
              tpTime[i] = char(154);  // 154 would result in a 3-digit t in overflow_it().
          
          overflow_it(tpTime, A15Result);
          
          str.assign(A15Result);
      }
  
      std::cout &lt;&lt; &quot;str says: &quot; &lt;&lt; str &lt;&lt; std::endl;
  
      return 0;
  }
  </code></pre>
  <p>Here are two facts we should remember before going on:
  1). My machine is an Intel x86 machine so it's using the Little Endian rule.  Therefore for a variable &quot;m&quot; of int type, whose value is, say, 10, it's memory layout might be like this:</p>
  <pre><code>Starting addr：0xbf89bebc: m(byte#1): 10
                 0xbf89bebd: m(byte#2): 0
                 0xbf89bebe: m(byte#3): 0
                 0xbf89bebf: m(byte#4): 0
  </code></pre>
  <p>2). The program above runs within the main thread.  When it comes to the overflow_it() function, the variables layout in the thread stack looks like this(which only shows the important variables):</p>
  <pre><code>0xbfc609e9 : temp[0]
  0xbfc609ea : temp[1]
  0xbfc609eb : temp[2]
  0xbfc609ec : m(byte#1) &lt;-- Note that m follows temp immediately.  m(byte#1) happens to be the byte temp[3].
  0xbfc609ed : m(byte#2)
  0xbfc609ee : m(byte#3)
  0xbfc609ef : m(byte#4)
  0xbfc609f0 : t
  ...(3 bytes)
  0xbfc609f4 : i
  ...(3 bytes)
  ...(etc. etc. etc...)
  0xbfc60a26 : A15Result  &lt;-- Data would be written to this buffer in overflow_it()
  ...(11 bytes)
  0xbfc60a32 : tpTime
  ...(5 bytes)
  0xbfc60a38 : str    &lt;-- Note the str takes up 4 bytes.  Its starting address is **16 bytes** behind A15Result.
  </code></pre>
  <p><strong>My analysis:</strong></p>
  <p>1). m is a counter in overflow_it() whose value is incremented by 1 at each for loop and whose max value is supposed not greater than 6.  Thus it's value could be stored completely in m(byte#1)(remember it's Little Endian) which happens to be temp<a href="http://gcc.gnu.org/onlinedocs/libstdc++/latest-doxygen/a00954_source.html" rel="nofollow noreferrer">3</a>.</p>
  <p>2). In the buggy line: When t is a 3-digit integer, such as 109, then the sprintf() call would result in a buffer overflow, because serializing the number 109 to the string &quot;109&quot; actually requires 4 bytes: '1', '0', '9' and a terminating '\0'.  Because temp[] is allocated with 3 bytes only, the final '\0' would definitely be written to temp<a href="http://gcc.gnu.org/onlinedocs/libstdc++/latest-doxygen/a00954_source.html" rel="nofollow noreferrer">3</a>, which is just the m(byte#1), which unfortunately stores m's value.  As a result, m's value is reset to 0 every time.</p>
  <p>3). The programmer's expectation, however, is that the for loop in the overflow_it() would execute 6 times only, with each time m being incremented by 1.  Because m is always reset to 0, the actual loop time is far more than 6 times.</p>
  <p>4). Let's look at the variable i in overflow_it(): Every time the for loop is executed, i's value is incremented by 2, and A15Result[i] will be accessed.  However, if you compile and run this program, you'll see the i value finally adds up to 24, which means the overflow_it() writes data to the bytes ranging from A15Result[0] to A15Result[23].  Note that the object str is only 16 bytes behind A15Result[0], thus the overflow_it() has &quot;sweeped through&quot; str and destroy it's correct memory layout.</p>
  <p>5). I think the correct use of std::string, as it is a non-POD data structure, depends on that that instantiated std::string object must have a correct internal state.  But in this program, str's internal layout has been changed by force externally.  This should be why the assign() method call would finally cause a segfault.</p>
  <hr />
  <p><strong>Update on 08/26/2011</strong></p>
  <p>In my previous update on 08/19/2011, I said that the segfault was caused by a method call on a local std::string object whose memory layout had been broken and thus became a &quot;destroyed&quot; object.  This is not an &quot;always&quot; true story.  Consider the C++ program below:</p>
  <pre><code>//C++
  class A {
      public:
          void Hello(const std::string&amp; name) {
             std::cout &lt;&lt; &quot;hello &quot; &lt;&lt; name;
           }
  };
  int main(int argc, char** argv)
  {
      A* pa = NULL; //!!
      pa-&gt;Hello(&quot;world&quot;);
      return 0;
  }
  </code></pre>
  <p>The Hello() call would succeed.  It would succeed even if you assign an obviously bad pointer to pa.  The reason is: the non-virtual methods of a class don't reside within the memory layout of the object, according to the C++ object model.  The C++ compiler turns the A::Hello() method to something like, say, A_Hello_xxx(A * const this, ...) which could be a global function.  Thus, as long as you don't operate on the &quot;this&quot; pointer, things could go pretty well.</p>
  <p>This fact shows that a &quot;bad&quot; object is <strong>NOT</strong> the root cause that results in the SIGSEGV segfault.  The assign() method is not virtual in std::string, thus the &quot;bad&quot; std::string object wouldn't cause the segfault.  There must be some other reason that finally caused the segfault.</p>
  <p>I noticed that the segfault comes from the __gnu_cxx::__exchange_and_add() function, so I then looked into its source code in <a href="http://gcc.gnu.org/onlinedocs/libstdc++/latest-doxygen/a00954_source.html" rel="nofollow noreferrer">this web page</a>:</p>
  <pre><code>00046   static inline _Atomic_word 
  00047   __exchange_and_add(volatile _Atomic_word* __mem, int __val)
  00048   { return __sync_fetch_and_add(__mem, __val); }
  </code></pre>
  <p>The __exchange_and_add() finally calls the __sync_fetch_and_add().  According to <a href="http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html" rel="nofollow noreferrer">this web page</a>, the __sync_fetch_and_add() is a GCC builtin function whose behavior is like this:</p>
  <pre><code>type __sync_fetch_and_add (type *ptr, type value, ...)
  {
      tmp = *ptr; 
      *ptr op= value; // Here the &quot;op=&quot; means &quot;+=&quot; as this function is &quot;_and_add&quot;.
      return tmp;
  }
  </code></pre>
  <p>There it is!  The passed-in ptr pointer is dereferenced here.  In the 08/19/2011 program, the ptr is actually the &quot;this&quot; pointer of the &quot;bad&quot; std::string object within the assign() method.  It is the derefenence at this point that actually caused the SIGSEGV segmentation fault.</p>
  <p>We could test this with the following program:</p>
  <pre><code>#include &lt;bits/atomicity.h&gt;
  
  int main(int argc, char * argv[])
  {
      __sync_fetch_and_add((_Atomic_word *)0, 10);    // Would result in a segfault.
  
      return 0;
  }
  </code></pre>