eR         & {\bf 39.2} & 59.7 & {\bf 42.7} & {\bf 35.6} & {\bf 56.7} & {\bf 38.2} & {\bf 66.1} & {\bf 87.3} & {\bf 72.1}\\
\end{tabular}
\caption{Results on mask rcnn on object detection, instance segmentation and keypoint detection fine-tuned on COCO. We show results on val2017, averaged over 5 trials.}
\label{table:maskrcnn}
\vspace{-3mm}
\end{table}

\vspace{-2mm}
\subsection{Ablation Studies}
\vspace{-.3cm}\paragraph{Influence of views.}
We analyze the importance of correctly matching pixels during training.
Table~\ref{table:ablation_view} shows results considering three pixel-matching strategies.
The first row (``unmatch'') ignores the correspondence map between views of the same scene. It considers random pairs of pixels as positive pairs.
Second and third rows utilize the correct correspondence map. The second row (``same view'') always consider identical crop for each pair of views, while the last (``diff. view'') consider different crops (with different location and scale). VADeR uses the third approach as default. Results are reported for semantic segmentation on Pascal VOC (in mIoU) and for dense correspondence on DAVIS-2017 (in region similarity $\mathcal{J}$).
As expected, using random pixel matching between views is worse than using correct pixel pairing (row 1 vs. rows 2 and 3).
We also note that performance in recognition task almost does not change when considering same or different crops between pairing views (row 2 vs. row 3). However, for correspondence, having difference crops per view provides a considerable advantage.

\vspace{-.3cm}\paragraph{Semantic grouping.}
Figure~\ref{fig:qualitative-pca} shows features learned by VADeR (big) and MoCo (small, bottom-right) projected to three dimensions (using PCA) and plotted as RGB. Similar colors implies that features are semantically similar. We observe qualitatively that \emph{semantically meaningful grouping emerges from VADeR training, without any supervision}. This is an interesting fact that can potentially be useful in unsupervised or non-parametric segmentation problems.

\begin{figure}[!t]
    \includegraphics[width=1\linewidth]{figures/qualitative-pca.pdf}
    \caption{Visualizing the learned features. We show query images, VADeR (big) and MoCo (small, bottom-right) features. The features are projected into three dimensions with PCA and visualized as RGB (similar to~\cite{vondrick2018tracking}). Similar color implies similarity in feature space. Best viewed in color.}
    \label{fig:qualitative-pca}
    \vspace{-3mm}
\end{figure}


\vspace{-.3cm}
\section{Conclusion}\label{sec:conclusion}
We present VADeR---View-Agnostic Dense Representations---for unsupervised learning of dense representations.
Our method learns representations through pixel-level contrastive learning.
VADeR is trained by forcing representations of matching pixels (that is, features from different views describing same location) to be close in an embedding space, while non-matching features to be far apart.
We leverage known pixel correspondences, derived from randomly generated views of a scene, to generate positive pairs.
Qualitatively examples hints that VADeR features can discovery high-level visual concepts and that semantic grouping can emerge from training, without any label.
We show that learning unsupervised dense representations are more efficient to downstream pixel-level tasks than their global counterparts. VADeR achieves positive results when compared to strong baselines in many structured prediction tasks, ranging from recognition to geometry.
We believe that learning unsupervised dense representations can be useful for many structured problems involved transfer learning, as well as unsupervised or low-data regime problems.

\newpage
\section{Broader Impact}
Our research falls under the category of advancing machine learning techniques for computer vision and scene understanding. We focus on improving image representations for dense prediction tasks, which subsumes a large array of fundamental vision tasks, such as image segmentation and object detection. While there are potentially many implications for using these applications, here we discuss two aspects. First, we highlight some social implications for image understanding with no or very little labeled data. Second, we provide some insights on foundational research questions regarding the evaluation of general purpose representation learning methods.

Improving capabilities of image understanding using unlabeled data, especially for pixel-level tasks, opens up a wide range of applications that are beneficial to the society, and which cannot be tackled otherwise. Medical imagery applications suffers from lack of labeled data due to the need of very specialized labelers. Another application, tackling harmful online content---including but not limited to terrorist propaganda, hateful speech, fake news and misinformation---is a huge challenge for governments and businesses. What makes these problems especially difficult is that it is very difficult to obtain clean labeled data for training machine learning models---think of filming a terrorist attack on live video as in the unfortunate Christchurch attack.
Self-supervised learning can potentially move the needle in advancing models for detecting extremely rare yet highly impactful incidents. On the other hand, such technologies can be potentially misused for violating privacy and freedom of expression. We acknowledge these risks as being a feature of any amoral technology, and we invite governments, policy makers and all citizens---including the research community---to work hard on striking a balance between those benefits and risks.

Another interesting aspect of our research is highlighting the importance of  aligning representation learning methods with the nature of downstream applications. With our method, we show that learning pixel-level representations from unlabeled data we can outperform image-level methods on a variety of dense prediction tasks. Our findings highlight that the research community should go beyond limited test-beds for evaluating generic representation learning techniques. We invite further research on developing comprehensive evaluation protocols for such methods. In fact, we see many research opportunities in the computer vision domain, such as developing a sweep of standardized benchmarks across a variety of geometric and semantic image understanding tasks, and designing methods that can bridge the gap between offline and online performance.

{\small
In the four films he's directed, Oren Moverman has mounted what feels like a career-long inquiry into the lost soul of America. That may be a crushingly pretentious thing to say about a filmmaker who shuffles moods, genres and visual palettes with enviable ease, and who can leap from the sun-streaked LAPD noir of "Rampart" (2011) to the gloomy Manhattan neorealism of "Time Out of Mind" (2014) without breaking a sweat.
But in each of those earlier pictures, and in his piercingly sad debut, "The Messenger" (2009), the director projected a chastening moral vision of a country that had on some fundamental level forsaken its own. He gave us despairing snapshots of men who found themselves on the outskirts of civilian society — a pair of soldiers tasked with notifying the bereaved, a dirty cop, a homeless man on the verge of fading away — and asked us to consider the world that had necessitated their particular strain of torment.
"The Dinner," Moverman's elegant and acerbic fourth feature, both continues and disrupts his style of social critique. Adapted and Americanized from the Dutch novelist Herman Koch's bestselling book (which has already inspired two earlier film versions), the movie has something to say about class struggle, racial tension and mental illness in this country, all couched in a withering portrait of the moneyed elite. Set over the course of one long evening, "The Dinner" turns a high-end double date into a seething evisceration of the upper class and the moral rot lurking beneath its well-heeled facade.
This is not a new or especially interesting dramatic thesis, and not even Moverman's ability to find nuance in the cracks of a familiar story can quite offset the sense that we are watching an artificial construct, a cleverly rigged parlor game that fancies itself a moral lament. You admire the skill and wit of the performers — a superb cast led by Steve Coogan, Laura Linney, Richard Gere and Rebecca Hall — in much the same way you might savor the movie's candlelit ambience. For all the actors' commitment and ferocity, the experience they offer feels less like a confrontation with the anxieties of modern life than a plush, moody escape.
The dinner in question has been called by Stan Lohman (Gere), a slick congressman and gubernatorial candidate who must put politics aside for one evening to deal with some unpleasant family business. He and his wife, Katelyn (Hall), join his abrasive brother, Paul (Coogan), and his wife, Claire (Linney), at the sort of posh dining establishment where the waiters serve up ornate postmodern culinary sculptures and painstakingly identify each ingredient by geographical origin.
Paul, a high-school history teacher with an advanced degree in misanthropy, spends a lot of time mocking the restaurant's pretensions, even as he strikes up an improbable friendship with the nervous but resourceful host (a charming Michael Chernus). He makes little secret of how much he resents Stan for his narcissism, his celebrity