<p dir="auto">The core memory allocation routines <code class="notranslate">fastMalloc()</code> and <code class="notranslate">fastFree()</code> are not thread-safe on startup due to a bug in a dependent function <code class="notranslate">isAlignedAllocationEnabled()</code> called by both.</p>
<p dir="auto">I have a PR to fix this. Earlier attempts to fix memory allocator startup behavior like issue <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="506218863" data-permission-text="Title is private" data-url="https://github.com/opencv/opencv/issues/15691" data-hovercard-type="issue" data-hovercard-url="/opencv/opencv/issues/15691/hovercard" href="https://github.com/opencv/opencv/issues/15691">#15691</a> and its PR are not themselves thread-safe. I request the team review my analysis below. When we agree the code is errant and undesired behavior, I will submit my fix PR. 👍</p>
<p dir="auto">I found this issue when I implemented native aligned memory on Windows. The fix for this issue is distinct from the native Windows aligned memory feature update. However, there is benefit both in reliability and speed to deploy this fix <em>before</em> native Windows aligned memory.</p>
<p dir="auto">This is a fail of the "static initialization order problem" <a href="https://isocpp.org/wiki/faq/ctors#static-init-order" rel="nofollow">https://isocpp.org/wiki/faq/ctors#static-init-order</a> and can be fixed with the "use construct on first use idiom" <a href="https://isocpp.org/wiki/faq/ctors#static-init-order-on-first-use" rel="nofollow">https://isocpp.org/wiki/faq/ctors#static-init-order-on-first-use</a></p>
<h3 dir="auto">System information (version)</h3>
<ul dir="auto">
<li>OpenCV =&gt; 4.5.0 or newer like master <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/opencv/opencv/commit/e726ff3296b9e382371a778ad807d37a2e49b979/hovercard" href="https://github.com/opencv/opencv/commit/e726ff3296b9e382371a778ad807d37a2e49b979"><tt>e726ff3</tt></a></li>
<li>Operating System / Platform =&gt; Windows 10 64-Bit</li>
<li>Compiler =&gt; Visual Studio Community 2019 v16.8.2</li>
</ul>
<h3 dir="auto">Steps to reproduce</h3>
<p dir="auto">It is more reliable to reproduce through code review. This is due to thread race conditions and that the errant code has few bad side-affects on Linux. On Windows it will crash (but you would need my native aligned memory update to expose this issue).</p>
<p dir="auto">Below I provide two scenarios and walk readers through them. Important things to remember...</p>
<ul dir="auto">
<li>C++ compilation units (.cpp files) are initialized at runtime in an undetermined order. There is <strong>no</strong> promise by the C++ runtime for order.</li>
<li>The thread names I give below <strong>do not</strong> imply any order. Thread A <strong>does not</strong> necessarily start before Thread B.</li>
<li>These threads are running in a multi-threaded OS on a CPU that has multiple cores which allows for simultaneous thread activity. This is normal for all modern OS and CPUs.</li>
<li>The rows do imply sequences. Top to bottom is the passage of time. Activities on the same row are happening at exactly the same time. This is possible because of multiple CPU cores on a multi-threaded OS.</li>
<li>The errant code is in the file <code class="notranslate">modules\core\src\alloc.cpp</code>. Refer to that file to follow the scenarios below.</li>
<li>For these scenarios, I declare that the runtime did <strong>not</strong> chose <code class="notranslate">alloc.cpp</code> compilation unit to initialize first. This means that <code class="notranslate">static const bool g_force_initialization_memalign_flag = isAlignedAllocationEnabled();</code> does <strong>not</strong> run in these scenarios.</li>
</ul>
<h4 dir="auto">Scenario 1</h4>
<table role="table">
<thead>
<tr>
<th>Time</th>
<th>thread A</th>
<th>thread B</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td></td>
<td>fastMalloc()</td>
</tr>
<tr>
<td>2</td>
<td> </td>
<td>isAlignedAllocationEnabled()</td>
</tr>
<tr>
<td>3</td>
<td>fastMalloc() </td>
<td>static bool initialized = false</td>
</tr>
<tr>
<td>4</td>
<td>isAlignedAllocationEnabled()</td>
<td>static bool useMemalign = true</td>
</tr>
<tr>
<td>5</td>
<td>if (!initialized)</td>
<td>if (!initialized)</td>
</tr>
<tr>
<td>6</td>
<td>initialized = true</td>
<td>initialized = true</td>
</tr>
<tr>
<td>7</td>
<td>useMemalign = readMemoryAlignmentParameter();</td>
<td>useMemalign = readMemoryAlignmentParameter();</td>
</tr>
<tr>
<td>8</td>
<td>…</td>
<td>…</td>
</tr>
</tbody>
</table>
<p dir="auto">You can see that there are two threads that are both in the initialization block of code. This is itself undesired. Next, both threads are calling <code class="notranslate">readMemoryAlignmentParameter()</code> and its many dependent functions. It is technically possible for <code class="notranslate">readMemoryAlignmentParameter()</code> and its dependencies to return different results. And therefore, it is possible <code class="notranslate">useMemalign</code> has different values in memory. Even the timing of the write to the memory location of <code class="notranslate">useMemalign</code> could lead to one thread overwriting the other thread's value.</p>
<p dir="auto">This is a coding error that probably has few bad side-affects. <code class="notranslate">readMemoryAlignmentParameter()</code> probably has no problems being called at the same time with the same parameters, and probably returns the same value. It is unlikely the environment variable it is reading is changed by a third thread while threads A and B are reading it. So in this scenario there is duplicate work but likely no critically bad side-affects.</p>
<h4 dir="auto">Scenario 2</h4>
<table role="table">
<thead>
<tr>
<th>time</th>
<th>thread C</th>
<th>thread D</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td></td>
<td>fastMalloc()</td>
</tr>
<tr>
<td>2</td>
<td> </td>
<td>isAlignedAllocationEnabled()</td>
</tr>
<tr>
<td>3</td>
<td> </td>
<td>static bool initialized = false</td>
</tr>
<tr>
<td>4</td>
<td> </td>
<td>static bool useMemalign = true</td>
</tr>
<tr>
<td>5</td>
<td>fastMalloc()</td>
<td>if (!initialized)</td>
</tr>
<tr>
<td>6</td>
<td>isAlignedAllocationEnabled()</td>
<td>initialized = true</td>
</tr>
<tr>
<td>7</td>
<td>if (!initialized)</td>
<td><em>thread delayed</em></td>
</tr>
<tr>
<td>8</td>
<td>return useMemalign <code class="notranslate">true</code></td>
<td><em>thread delayed</em></td>
</tr>
<tr>
<td>9</td>
<td>… memory is allocated using the aligned codepath</td>
<td><em>thread delayed</em></td>
</tr>
<tr>
<td>10</td>
<td>…</td>
<td>useMemalign = readMemoryAlignmentParameter() <code class="notranslate">false due to conditions or envvar</code></td>
</tr>
<tr>
<td>11</td>
<td>…</td>
<td>return useMemalign <code class="notranslate">false</code></td>
</tr>
<tr>
<td>12</td>
<td>…</td>
<td>… memory is allocated using the un-aligned codepath</td>
</tr>
<tr>
<td>13</td>
<td>fastFree()</td>
<td>fastFree()</td>
</tr>
<tr>
<td>14</td>
<td>if (isAlignedAllocationEnabled()) <code class="notranslate">this is false</code></td>
<td>if (isAlignedAllocationEnabled()) <code class="notranslate">this is false</code></td>
</tr>
<tr>
<td>15</td>
<td>… <g-emoji class="g-emoji" alias="boom" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f4a5.png">💥</g-emoji>unalignedfree(aligned memory pointer)</td>
<td>… unalignedfree(un-aligned memory pointer)</td>
</tr>
</tbody>
</table>
<p dir="auto">This scenario has bad side-affects and potentially program crashes. The current codebase often hides this errant code because <code class="notranslate">free()</code> is used both for native-aligned and native-<em>un</em>aligned memory. Also, the timing (like scenario 1) may be such that the fallback aligning index[-1] approach isn't allocating/free incompatible blocks of memory while initialization is occuring. But this more likely to fail on Windows because Windows has distinct functions <code class="notranslate">free()</code> and <code class="notranslate">_aligned_free()</code>.</p>
<p dir="auto">Follow this...</p>
<ul dir="auto">
<li>time 8: thread C gets <code class="notranslate">true</code> from <code class="notranslate">isAlignedAllocationEnabled()</code> because thread D set <code class="notranslate">useMemalign=true</code> and has not yet completed the initialization block that will eventually set it to <code class="notranslate">false</code></li>
<li>time 9: thread C gets memory allocated using the <em>aligned</em> codepath</li>
<li>time 11: thread D gets <code class="notranslate">false</code> from <code class="notranslate">isAlignedAllocationEnabled()</code></li>
<li>time 12: thread D gets memory allocated using <em>un-aligned</em> codepath</li>
<li>time 14: both threads get <code class="notranslate">false</code> from <code class="notranslate">isAlignedAllocationEnabled()</code></li>
<li>time 15: both threads use the <em>un-aligned</em> memory deallocator</li>
</ul>
<p dir="auto">The fault occurs at time 15.<br>
Thread C uses the un-aligned memory deallocator on aligned memory. ☹<br>
Thread D uses the un-aligned memory deallocator on un-aligned memory.</p>
<h3 dir="auto">Fix</h3>
<p dir="auto">The fix is to use the classic "Construct On First Use Idiom" as documented by our ISO friends at <a href="https://isocpp.org/wiki/faq/ctors#static-init-order-on-first-use" rel="nofollow">https://isocpp.org/wiki/faq/ctors#static-init-order-on-first-use</a>. The specific approach in my PR might also be unmeasurably faster as it has fewer if() tests and fewer variables.</p>
<h3 dir="auto">Issue submission checklist</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I report the issue, it's not a question</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I checked the problem with documentation, FAQ, open issues,<br>
answers.opencv.org, Stack Overflow, etc and have not found solution</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I updated to latest OpenCV version and the issue is still there</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> There is reproducer code and related data files: videos, images, onnx, etc</li>
</ul>
<h3 dir="auto">Please state the information for your system</h3>
<p dir="auto">OpenCV version: 3.1 (verified with 3.0)<br>
Host OS: Windows 7 and Windows Server 2012R2<br>
Target OS: ARM board<br>
Compiler &amp; CMake: GCC 4.9.3 (linaro 2015.05) &amp; CMake 3.4.1</p>
<h3 dir="auto">In which part of the OpenCV library you got the issue?</h3>
<p dir="auto">Cross-building of core</p>
<h3 dir="auto">Expected behaviour</h3>
<p dir="auto">Build works irrespective of msys environment</p>
<h3 dir="auto">Actual behaviour</h3>
<p dir="auto">Build works fine using the much older msys1 (make 3.81 / 2006) but fails with msys2 (make 4.1-4 / 2014)</p>
<h3 dir="auto">Additional description</h3>
<p dir="auto">building with msys2 fails while attempting to generate precomp.hpp.</p>
<h3 dir="auto">Code example to reproduce the issue / Steps to reproduce the issue</h3>
<p dir="auto">Configuration:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="c:\jenkins\tools\hudson.plugins.cmake.CmakeTool\3.4.1\bin\cmake -G &quot;MSYS Makefiles&quot; -D CMAKE_BUILD_TYPE=Release -Wno-dev -DCMAKE_INSTALL_PREFIX=c:\jenkins\workspace\libopencv\838c53df -DCMAKE_TOOLCHAIN_FILE=c:\jenkins\workspace\libopencv\838c53df/cmake-scripts/arm-linaro.cmake -DEXECUTABLE_OUTPUT_PATH=c:/jenkins/workspace/libopencv/838c53df/bin -DWITH_CUDA=OFF -DWITH_OPENEXR=OFF -DWITH_TIFF=OFF -DBUILD_TIFF=OFF -DWITH_JPEG=OFF -DBUILD_JPEG=OFF -DWITH_PNG=OFF -DBUILD_PNG=OFF -DWITH_WEBP=OFF -DBUILD_WEBP=OFF -DWITH_FFMPEG=OFF -DWITH_IPP=OFF -DWITH_JASPER=OFF -DBUILD_JASPER=OFF -DWITH_VTK=OFF -DWITH_V4L=OFF -DWITH_OPENCL=OFF -DBUILD_opencv_video=OFF -DBUILD_opencv_videostab=OFF -DBUILD_opencv_videolo=OFF -DBUILD_opencv_apps=OFF -DBUILD_opencv_stitching=OFF -DBUILD_DOCS=OFF -DBUILD_JASPER=OFF -DBUILD_TESTS=ON -DINSTALL_TESTS=ON -DBUILD_PERF_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DBUILD_TBB=OFF -DEIGEN_INCLUDE_PATH=c:\jenkins\workspace\libopencv\838c53df/libeigen c:\jenkins\workspace\libopencv\838c53df\repository"><pre class="notranslate"><code class="notranslate">c:\jenkins\tools\hudson.plugins.cmake.CmakeTool\3.4.1\bin\cmake -G "MSYS Makefiles" -D CMAKE_BUILD_TYPE=Release -Wno-dev -DCMAKE_INSTALL_PREFIX=c:\jenkins\workspace\libopencv\838c53df -DCMAKE_TOOLCHAIN_FILE=c:\jenkins\workspace\libopencv\838c53df/cmake-scripts/arm-linaro.cmake -DEXECUTABLE_OUTPUT_PATH=c:/jenkins/workspace/libopencv/838c53df/bin -DWITH_CUDA=OFF -DWITH_OPENEXR=OFF -DWITH_TIFF=OFF -DBUILD_TIFF=OFF -DWITH_JPEG=OFF -DBUILD_JPEG=OFF -DWITH_PNG=OFF -DBUILD_PNG=OFF -DWITH_WEBP=OFF -DBUILD_WEBP=OFF -DWITH_FFMPEG=OFF -DWITH_IPP=OFF -DWITH_JASPER=OFF -DBUILD_JASPER=OFF -DWITH_VTK=OFF -DWITH_V4L=OFF -DWITH_OPENCL=OFF -DBUILD_opencv_video=OFF -DBUILD_opencv_videostab=OFF -DBUILD_opencv_videolo=OFF -DBUILD_opencv_apps=OFF -DBUILD_opencv_stitching=OFF -DBUILD_DOCS=OFF -DBUILD_JASPER=OFF -DBUILD_TESTS=ON -DINSTALL_TESTS=ON -DBUILD_PERF_TESTS=OFF -DBUILD_SHARED_LIBS=OFF -DBUILD_TBB=OFF -DEIGEN_INCLUDE_PATH=c:\jenkins\workspace\libopencv\838c53df/libeigen c:\jenkins\workspace\libopencv\838c53df\repository
</code></pre></div>
<p dir="auto">and running the build with</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="c:\jenkins\tools\hudson.plugins.cmake.CmakeTool\3.4.1\bin\cmake --build c:\jenkins\workspace\libopencv\838c53df\jenkins-build --target install --config Release"><pre class="notranslate"><code class="notranslate">c:\jenkins\tools\hudson.plugins.cmake.CmakeTool\3.4.1\bin\cmake --build c:\jenkins\workspace\libopencv\838c53df\jenkins-build --target install --config Release
</code></pre></div>
<p dir="auto">This generates the offending command:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cd /C/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core &amp;&amp; /c/jenkins/tools/hudson.plugins.cmake.CmakeTool/3.4.1/bin/cmake.exe -E make_directory C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp.gch
cd /C/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core &amp;&amp; /C/linaro/gcc-4.9-2015.05-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-g++.exe -O3 -DNDEBUG -DNDEBUG &quot;-D_ARM&quot; &quot;-D_ARM&quot; &quot;-D__OPENCV_BUILD=1&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/libeigen&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/libeigen&quot; -I&quot;C:/jenkins/workspace/libopencv/838c53df/repository/modules/core/include&quot; -I&quot;C:/jenkins/workspace/libopencv/838c53df/repository/modules/core/src&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core&quot; -I&quot;C:/jenkins/workspace/libopencv/838c53df/repository/modules/core/include&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/repository/3rdparty/zlib&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build/3rdparty/zlib&quot; -std=c++11 -O3 -DNDEBUG -mvectorize-with-neon-quad -mcpu=cortex-a9 -mfloat-abi=softfp -mfpu=neon -ffast-math -funroll-loops -fomit-frame-pointer -mthumb -fdata-sections -fsigned-char -Wno-psabi -mthumb -fdata-sections -fsigned-char -Wno-psabi -fPIC -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -x c++-header -o C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp.gch/opencv_core_Release.gch C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp
C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp:46:38: fatal error: opencv2/opencv_modules.hpp: No such file or directory
 #include &quot;opencv2/opencv_modules.hpp&quot;"><pre class="notranslate"><code class="notranslate">cd /C/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core &amp;&amp; /c/jenkins/tools/hudson.plugins.cmake.CmakeTool/3.4.1/bin/cmake.exe -E make_directory C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp.gch
cd /C/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core &amp;&amp; /C/linaro/gcc-4.9-2015.05-x86_64_arm-linux-gnueabi/bin/arm-linux-gnueabi-g++.exe -O3 -DNDEBUG -DNDEBUG "-D_ARM" "-D_ARM" "-D__OPENCV_BUILD=1" -isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build" -isystem"C:/jenkins/workspace/libopencv/838c53df/libeigen" -isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build" -isystem"C:/jenkins/workspace/libopencv/838c53df/libeigen" -I"C:/jenkins/workspace/libopencv/838c53df/repository/modules/core/include" -I"C:/jenkins/workspace/libopencv/838c53df/repository/modules/core/src" -isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core" -I"C:/jenkins/workspace/libopencv/838c53df/repository/modules/core/include" -isystem"C:/jenkins/workspace/libopencv/838c53df/repository/3rdparty/zlib" -isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build/3rdparty/zlib" -std=c++11 -O3 -DNDEBUG -mvectorize-with-neon-quad -mcpu=cortex-a9 -mfloat-abi=softfp -mfpu=neon -ffast-math -funroll-loops -fomit-frame-pointer -mthumb -fdata-sections -fsigned-char -Wno-psabi -mthumb -fdata-sections -fsigned-char -Wno-psabi -fPIC -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -x c++-header -o C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp.gch/opencv_core_Release.gch C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp
C:/jenkins/workspace/libopencv/838c53df/jenkins-build/modules/core/precomp.hpp:46:38: fatal error: opencv2/opencv_modules.hpp: No such file or directory
 #include "opencv2/opencv_modules.hpp"
</code></pre></div>
<p dir="auto">the problem being that the build system generates two pairs of -isystem includes for the project root instead of generating one -isystem and one standard -I include pair:</p>
<p dir="auto">generated include statements (extracted from above, breaks the build):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="-isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/libeigen&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/libeigen&quot; "><pre class="notranslate"><code class="notranslate">-isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build" -isystem"C:/jenkins/workspace/libopencv/838c53df/libeigen" -isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build" -isystem"C:/jenkins/workspace/libopencv/838c53df/libeigen" 
</code></pre></div>
<p dir="auto">manually corrected include statements (fixes the build):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="-isystem&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build&quot; -isystem&quot;C:/jenkins/workspace/libopencv/838c53df/libeigen&quot; -I&quot;C:/jenkins/workspace/libopencv/838c53df/jenkins-build&quot; -I&quot;C:/jenkins/workspace/libopencv/838c53df/libeigen&quot; "><pre class="notranslate"><code class="notranslate">-isystem"C:/jenkins/workspace/libopencv/838c53df/jenkins-build" -isystem"C:/jenkins/workspace/libopencv/838c53df/libeigen" -I"C:/jenkins/workspace/libopencv/838c53df/jenkins-build" -I"C:/jenkins/workspace/libopencv/838c53df/libeigen" 
</code></pre></div>
<p dir="auto">Everything else being the same, this build runs fine with msys1. The infrastructure however runs on msys2. I would very much appreciate a hint on how to fix or workaround the issue.</p>