<h2 dir="auto">Issue description</h2>
<p dir="auto">I noticed that some loss functions in <code class="notranslate">torch.nn.functional</code> do tensor broadcasting while other does not.<br>
I started using <code class="notranslate">cosine_embedding_loss</code> and was happy because it handles input tensors with (N, F) and (1, F) shapes, where N - number of examples, F - number of features in one example (vector). Then I moved to <code class="notranslate">mse_loss</code> and <code class="notranslate">l1_loss</code> to check for better results. And I was disappointed to see that the latter both require (N, F) and (N, F) tensors - they do not support tensor broadcasting as <code class="notranslate">cosine_embedding_loss</code> does. Now I need to either stack the second tensor N times (and I do not know whether the gradient history is stored if a tensor is copied; in any case it is memory and computationaly inefficient) or implement l1 and l2 losses on my own.</p>
<p dir="auto">Please, make loss behavior consistent - either all functions require exactly the same shape or handle broadcasting for each of them. Thank you.</p>
<h2 dir="auto">Code example</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="t1 = torch.randn(5,7)
t2 = torch.randn(1,7)
loss_cosine = F.cosine_embedding_loss(t1, t2, target=torch.ones(5))  # works fine
loss_l2 = F.mse_loss(t1, t2)
# RuntimeError: input and target shapes do not match: input [5 x 7], target [1 x 7]"><pre class="notranslate"><code class="notranslate">t1 = torch.randn(5,7)
t2 = torch.randn(1,7)
loss_cosine = F.cosine_embedding_loss(t1, t2, target=torch.ones(5))  # works fine
loss_l2 = F.mse_loss(t1, t2)
# RuntimeError: input and target shapes do not match: input [5 x 7], target [1 x 7]
</code></pre></div>
<ul dir="auto">
<li>How you installed PyTorch: <code class="notranslate">conda install pytorch-cpu torchvision-cpu -c pytorch</code></li>
<li>PyTorch version: 0.4.1</li>
<li>Python version: 3.6.6</li>
</ul>
<h2 dir="auto">To Reproduce</h2>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="a = T.zeros(5,1,10, requires_grad=True)
b = T.zeros(5,10,10, requires_grad=True)
F.mse_loss(a,b)
# OK

a = T.zeros(5,1,10)
b = T.zeros(5,10,10)
F.mse_loss(a,b)
# RuntimeError: input and target shapes do not match: input [5 x 1 x 10], target [5 x 10 x 10] at /opt/conda/conda-bld/pytorch_1524577523076/work/aten/src/THNN/generic/MSECriterion.c:13"><pre class="notranslate"><span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-v">T</span>.<span class="pl-en">zeros</span>(<span class="pl-c1">5</span>,<span class="pl-c1">1</span>,<span class="pl-c1">10</span>, <span class="pl-s1">requires_grad</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
<span class="pl-s1">b</span> <span class="pl-c1">=</span> <span class="pl-v">T</span>.<span class="pl-en">zeros</span>(<span class="pl-c1">5</span>,<span class="pl-c1">10</span>,<span class="pl-c1">10</span>, <span class="pl-s1">requires_grad</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
<span class="pl-v">F</span>.<span class="pl-en">mse_loss</span>(<span class="pl-s1">a</span>,<span class="pl-s1">b</span>)
<span class="pl-c"># OK</span>

<span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-v">T</span>.<span class="pl-en">zeros</span>(<span class="pl-c1">5</span>,<span class="pl-c1">1</span>,<span class="pl-c1">10</span>)
<span class="pl-s1">b</span> <span class="pl-c1">=</span> <span class="pl-v">T</span>.<span class="pl-en">zeros</span>(<span class="pl-c1">5</span>,<span class="pl-c1">10</span>,<span class="pl-c1">10</span>)
<span class="pl-v">F</span>.<span class="pl-en">mse_loss</span>(<span class="pl-s1">a</span>,<span class="pl-s1">b</span>)
<span class="pl-c"># RuntimeError: input and target shapes do not match: input [5 x 1 x 10], target [5 x 10 x 10] at /opt/conda/conda-bld/pytorch_1524577523076/work/aten/src/THNN/generic/MSECriterion.c:13</span></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">Broadcast should work the same as with grad.</p>
<h2 dir="auto">Workaround</h2>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="# assumes size_average=False
def my_mse(a,b):
    return ((a-b)**2).sum()"><pre class="notranslate"><span class="pl-c"># assumes size_average=False</span>
<span class="pl-k">def</span> <span class="pl-en">my_mse</span>(<span class="pl-s1">a</span>,<span class="pl-s1">b</span>):
    <span class="pl-k">return</span> ((<span class="pl-s1">a</span><span class="pl-c1">-</span><span class="pl-s1">b</span>)<span class="pl-c1">**</span><span class="pl-c1">2</span>).<span class="pl-en">sum</span>()</pre></div>