Why does Python's cProfile report a different elapsed time than using time.time() deltas when using PyTorch?
I am profiling some code using PyTorch. I am aware the CUDA normally has some asynchronous execution (see PyTorch docs), but I believe that transferring from GPU to CPU will generally force synchronization.
For this reason, I decided to naively profile using cProfile, but I notice that the time reported by Profile.enable() ... Profile.disable() was different than the time recorded across time.time() (as a delta).
Here's what the code looks like at a high-level:
gpu = torch.device(&quot;cuda&quot;)
cpu = torch.device(&quot;cpu&quot;)

setup = Setup()
net = make_fcn_resnet50(num_classes=setup.D)
net.eval().to(gpu)

rgb_tensor = setup.sample(device=cpu)

pr = profile.Profile()
pr.enable()
t_start = time.time()

rgb_tensor = rgb_tensor.to(gpu)
y = net(rgb_tensor)
dd_tensor = y[&quot;out&quot;]
dd_mean = torch.mean(dd_tensor[[0]]).to(cpu).numpy()
assert dd_mean is not None

dt = time.time() - t_start
pr.disable()
stats = pstats.Stats(pr)

stats.print_stats(5)
print(f&quot;dt: {dt:.4f}s&quot;)

Here's the discrepancy I see:
2925 function calls (2734 primitive calls) in 0.009 seconds
...
dt: 0.0355s

I would've expected cProfile to report about 35ms (same as dt), but instead it reports about 10ms.
Why does this happen?
Full code + repro is here: 
https://github.com/EricCousineau-TRI/repro/tree/bdef8a14/python/cprofile_with_torch

Empirically, it seems like cProfile will not &quot;hook&quot; into the code if you don't &quot;flush&quot; all of the outputs, or if you're code isn't fully wrapped in a function.
See more detail in the comments here: 
https://github.com/EricCousineau-TRI/repro/blob/bdef8a14b5/python/cprofile_with_torch/repro.py#L75-L94
All timing results recorded with:

Ubuntu 18.04
CPython 3.6.9
nvidia-driver-450 (450.102.04-0ubuntu0.18.04.1)
NVidia Titan RTX


That all being said, may be better just to use PyTorch's provided mechanisms (:facepalm:):

https://pytorch.org/tutorials/beginner/profiler.html
https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html

