<p dir="auto">Hi,<br>
my task is: <code class="notranslate">make the first run of X as fast as the second run</code>.<br>
It seems quite difficult to achieve.<br>
The situation is, that I'm covering all code I want to execute (over runtests.jl), so in theory it should be straight forward to infer what to compile.</p>
<p dir="auto">What I've tried:</p>
<ol dir="auto">
<li>Including runtests.jl into user image</li>
<li><a href="https://github.com/timholy/SnoopCompile.jl">SnoopCompile</a> + user image</li>
</ol>
<div class="highlight highlight-source-julia notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="use SnoopCompile to generate snoop_precompiles.jl from runtests.jl
---- MyPackage.jl
module MyPackage
include(&quot;snoop_precompiles.jl&quot;)
end
---- userimg.jl
using MyPacke
----"><pre class="notranslate">use SnoopCompile to generate snoop_precompiles<span class="pl-k">.</span>jl from runtests<span class="pl-k">.</span>jl
<span class="pl-k">----</span> MyPackage<span class="pl-k">.</span>jl
<span class="pl-k">module</span> MyPackage
<span class="pl-c1">include</span>(<span class="pl-s"><span class="pl-pds">"</span>snoop_precompiles.jl<span class="pl-pds">"</span></span>)
<span class="pl-k">end</span>
<span class="pl-k">----</span> userimg<span class="pl-k">.</span>jl
<span class="pl-k">using</span> MyPacke
<span class="pl-k">----</span></pre></div>
<ol start="3" dir="auto">
<li><a href="https://github.com/dhoegh/BuildExecutable.jl">BuildExecutable</a></li>
</ol>
<p dir="auto">Results:</p>
<ol dir="auto">
<li>I have had good results with this approach, but it doesn't work for arbitrary code. Would it be worth while to trim the included code to work? I had various failures, and as soon as OpenGL comes into the game, things were not working at all anymore</li>
<li>This sometimes generates a compilation speed up, sometimes not... Sometimes it changes type inference resulting in weird errors. Is this SnoopCompiles fault, or to be expected? Would fixing SnoopCompile be well invested time?</li>
<li>The compiled executable actually executes slower than <code class="notranslate">julia executable_script.jl</code>, suggesting that this just links the exe to the JIT, while not caching any binary. I guess I still need precompile statements and it doesn't recursively infer from <code class="notranslate">main()</code> what to compile?</li>
</ol>
<p dir="auto">My dream workflow is something like this:</p>
<div class="highlight highlight-source-julia notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="# this could be executed by Pkg.build() -&gt;
&gt;julia --cache-binary --output-incremental=yes GLVisualize/test/runtests.jl --&gt; v0.5/lib/cache_x.so
julia&gt; using GLVisualize; &lt;-- all functions already run in runtests will need no JIT
&gt;generate_py_wrapper v0.5/lib/cache_x.so GLVisualize/src/GLVisualize.jl
--&gt; generates py c interop thingies to call the lib... Hehehe"><pre class="notranslate"><span class="pl-c"><span class="pl-c">#</span> this could be executed by Pkg.build() -&gt;</span>
<span class="pl-k">&gt;</span>julia <span class="pl-k">--</span>cache<span class="pl-k">-</span>binary <span class="pl-k">--</span>output<span class="pl-k">-</span>incremental<span class="pl-k">=</span>yes GLVisualize<span class="pl-k">/</span>test<span class="pl-k">/</span>runtests<span class="pl-k">.</span>jl <span class="pl-k">--&gt;</span> v0.<span class="pl-c1">5</span><span class="pl-k">/</span>lib<span class="pl-k">/</span>cache_x<span class="pl-k">.</span>so
julia<span class="pl-k">&gt;</span> <span class="pl-k">using</span> GLVisualize; <span class="pl-k">&lt;-</span><span class="pl-k">-</span> all functions already run <span class="pl-k">in</span> runtests will need no JIT
<span class="pl-k">&gt;</span>generate_py_wrapper v0.<span class="pl-c1">5</span><span class="pl-k">/</span>lib<span class="pl-k">/</span>cache_x<span class="pl-k">.</span>so GLVisualize<span class="pl-k">/</span>src<span class="pl-k">/</span>GLVisualize<span class="pl-k">.</span>jl
<span class="pl-k">--&gt;</span> generates py c interop thingies to call the lib<span class="pl-k">...</span> Hehehe</pre></div>
<p dir="auto">Btw, would generating a seperate .so + ccall solve the (quite annoying) problem, that we always need to interact with the sysimg?</p>
<p dir="auto">I'd be quite willing to work on this in some way.<br>
I just need to know where to start :)</p>
<p dir="auto">At this point it makes sense to separate <code class="notranslate">julia</code> into an interactive <code class="notranslate">julia</code> which runs scripts and starts the REPL and <code class="notranslate">julia-compile</code> which provides various bootstrapping and compilation features. The basic usage of <code class="notranslate">julia-compile</code> should be <code class="notranslate">julia-compile program.jl</code> which should produce an executable called <code class="notranslate">program</code> which, when run behaves the same way that <code class="notranslate">julia program.jl</code> would. In more advanced modes, it can behave like the current <code class="notranslate">julia --compile=all</code>, etc. Let's use this issue to discuss the desired API and how to split up the options.</p>