\section{Template Overview}
As noted in the introduction, the ``\verb|acmart|'' document class can be used to prepare many different kinds of documentation --- a double-blind initial submission of a full-length technical paper, a two-page SIGGRAPH Emerging Technologies abstract, a ``camera-ready'' journal article, a SIGCHI Extended Abstract, and more --- all by selecting the appropriate {\it template style} and {\it template parameters}.

This document will explain the major features of the document class. For further information, the {\it \LaTeX\ User's Guide} is available from \url{https://www.acm.org/publications/proceedings-template}.

\subsection{Template Styles}

The primary parameter given to the ``\verb|acmart|'' document class is the {\it template style} which corresponds to the kind of publication or SIG publishing the work. This parameter is enclosed in square brackets and is a part of the {\verb|documentclass|} command:
\begin{verbatim}
  \documentclass[STYLE]{acmart}
\end{verbatim}

Journals use one of three template styles. All but three ACM journals use the {\verb|acmsmall|} template style:
\begin{itemize}
\item {\verb|acmsmall|}: The default journal template style.
\item {\verb|acmlarge|}: Used by JOCCH and TAP.
\item {\verb|acmtog|}: Used by TOG.
\end{itemize}

The majority of conference proceedings documentation will use the {\verb|acmconf|} template style.
\begin{itemize}
\item {\verb|acmconf|}: The default proceedings template style.
\item{\verb|sigchi|}: Used for SIGCHI conference articles.
\item{\verb|sigchi-a|}: Used for SIGCHI ``Extended Abstract'' articles.
\item{\verb|sigplan|}: Used for SIGPLAN conference articles.
\end{itemize}

\subsection{Template Parameters}

In addition to specifying the {\it template style} to be used in formatting your work, there are a number of {\it template parameters} which modify some part of the applied template style. A complete list of these parameters can be found in the {\it \LaTeX\ User's Guide.}

Frequently-used parameters, or combinations of parameters, include:
\begin{itemize}
\item {\verb|anonymous,review|}: Suitable for a ``double-blind'' conference submission. Anonymizes the work and includes line numbers. Use with the \verb|\acmSubmissionID| command to print the submission's unique ID on each page of the work.
\item{\verb|authorversion|}: Produces a version of the work suitable for posting by the author.
\item{\verb|screen|}: Produces colored hyperlinks.
\end{itemize}

This document uses the following string as the first command in the source file: \verb|\documentclass[sigconf,screen]{acmart}|.

\section{Modifications}

Modifying the template --- including but not limited to: adjusting margins, typeface sizes, line spacing, paragraph and list definitions, and the use of the \verb|\vspace| command to manually adjust the vertical spacing between elements of your work --- is not allowed.

{\bf Your document will be returned to you for revision if modifications are discovered.}

\section{Typefaces}

The ``\verb|acmart|'' document class requires the use of the ``Libertine'' typeface family. Your \TeX\ installation should include this set of packages. Please do not substitute other typefaces. The ``\verb|lmodern|'' and ``\verb|ltimes|'' packages should not be used, as they will override the built-in typeface families.

\section{Title Information}

The title of your work should use capital letters appropriately - \url{https://capitalizemytitle.com/} has useful rules for capitalization. Use the {\verb|title|} command to define the title of your work. If your work has a subtitle, define it with the {\verb|subtitle|} command.
Do not insert line breaks in your title.

If your title is lengthy, you must define a short version to be used in the page headers, to prevent overlapping text. The \verb|title| command has a ``short title'' parameter:
\begin{verbatim}
  \title[short title]{full title}
\end{verbatim}

\section{Authors and Affiliations}

Each author must be defined separately for accurate metadata identification. Multiple authors may share one affiliation. Authors' names should not be abbreviated; use full first names wherever possible. Include authors' e-mail addresses whenever possible.

Grouping authors' names or e-mail addresses, or providing an ``e-mail alias,'' as shown below, is not acceptable:
\begin{verbatim}
  \author{Brooke Aster, David Mehldau}
  \email{dave,judy,steve@university.edu}
  \email{firstname.lastname@phillips.org}
\end{verbatim}

The \verb|authornote| and \verb|authornotemark| commands allow a note to apply to multiple authors --- for example, if the first two authors of an article contributed equally to the work. 

If your author list is lengthy, you must define a shortened version of the list of authors to be used in the page headers, to prevent overlapping text. The following command should be placed just after the last \verb|\author{}| definition:
\begin{verbatim}
  \renewcommand{\shortauthors}{McCartney, et al.}
\end{verbatim}
Omitting this command will force the use of a concatenated list of all of the authors' names, which may result in overlapping text in the page headers.

The article template's documentation, available at  \url{https://www.acm.org/publications/proceedings-template}, has a complete explanation of these commands and tips for their effective use.

\section{Rights Information}

Authors of any work published by ACM will need to complete a rights form. Depending on the kind of work, and the rights management choice made by the author, this may be copyright transfer, permission, license, or an OA (open access) agreement.

Regardless of the rights management choice, the author will receive a copy of the completed rights form once it has been submitted. This form contains \LaTeX\ commands that must be copied into the source document. When the document source is compiled, these commands and their parameters add formatted text to several areas of the final document:
\begin{itemize}
\item the ``ACM Reference Format'' text on the first page.
\item the ``rights management'' text on the first page.
\item the conference information in the page header(s).
\end{itemize}

Rights information is unique to the work; if you are preparing several works for an event, make sure to use the correct set of commands with each of the works.

\section{CCS Concepts and User-Defined Keywords}

Two elements of the ``acmart'' document class provide powerful taxonomic tools for you to help readers find your work in an online search. 

The ACM Computing Classification System --- \url{https://www.acm.org/publications/class-2012} --- is a set of classifiers and concepts that describe the computing discipline. Authors can select entries from this classification system, via \url{https://dl.acm.org/ccs/ccs.cfm}, and generate the commands to be included in the \LaTeX\ source. 

User-defined keywords are a comma-separated list of words and phrases of the authors' choosing, providing a more flexible way of describing the research being presented.

CCS concepts and user-defined keywords are required for all short- and full-length articles, and optional for two-page abstracts. 

\section{Sectioning Commands}

Your work should use standard \LaTeX\ sectioning commands: \verb|section|, \verb|subsection|, \verb|subsubsection|, and \verb|paragraph|. They should be numbered; do not remove the numbering from the commands. 

Simulating a sectioning command by setting the first word or words of a paragraph in boldface or italicized text is {\bf not allowed.}

\section{Tables}

The ``\verb|acmart|'' document class includes the ``\verb|booktabs|'' package --- \url{https://ctan.org/pkg/booktabs} --- for preparing high-quality tables. 

Table captions are placed {\it above} the table.

Because tables cannot be split across pages, the best placement for them is typically the top of the page nearest their initial cite.  To ensure this proper ``floating'' placement of tables, use the environment \textbf{table} to enclose the table's contents and the table caption.  The contents of the table itself must go in the \textbf{tabular} environment, to be aligned properly in rows and columns, with the desired horizontal and vertical rules.  Again, detailed instructions on \textbf{tabular} material are found in the \textit{\LaTeX\ User's Guide}.

Immediately following this sentence is the point at which Table~\ref{tab:freq} is included in the input file; compare the placement of the table here with the table in the printed output of this document.

\begin{table}
  \caption{Frequency of Special Characters}
  \label{tab:freq}
  \begin{tabular}{ccl}
    \toprule
    Non-English or Math&Frequency&Comments\\
    \midrule
    \O & 1 in 1,000& For Swedish names\\
    $\pi$ & 1 in 5& Common in math\\
    \$ & 4 in 5 & Used in business\\
    $\Psi^2_1$ & 1 in 40,000& Unexplained usage\\
  \bottomrule
\end{tabular}
\end{table}

To set a wider table, which takes up the whole width of the page's live area, use the environment \textbf{table*} to enclose the table's contents and the table caption.  As with a single-column table, this wide table will ``float'' to a location deemed more desirable. Immediately following this sentence is the point at which Table~\ref{tab:commands} is included in the input file; again, it is instructive to compare the placement of the table here with the table in the printed output of this document.

\begin{table*}
  \caption{Some Typical Commands}
  \label{tab:commands}
  \begin{tabular}{ccl}
    \toprule
    Command &A Number & Comments\\
    \midrule
    \texttt{{\char'134}author} & 100& Author \\
    \texttt{{\char'134}table}& 300 & For tables\\
    \texttt{{\char'134}table*}& 400& For wider tables\\
    \bottomrule
  \end{tabular}
\end{table*}

\section{Math Equations}
You may want to display math equations in three distinct styles:
inline, numbered or non-numbered display.  Each of
the three are discussed in the next sections.

\subsection{Inline (In-text) Equations}
A formula that appears in the running text is called an
inline or in-text formula.  It is produced by the
\textbf{math} environment, which can be
invoked with the usual \texttt{{\char'134}begin\,\ldots{\char'134}end}
construction or with the short form \texttt{\$\,\ldots\$}. You
can use any of the symbols and structures,
from $\alpha$ to $\omega$, available in
\LaTeX~\cite{Lamport:LaTeX}; this section will simply show a
few examples of in-text equations in context. Notice how
this equation:
\begin{math}
  \lim_{n\rightarrow \infty}x=0
\end{math},
set here in in-line math style, looks slightly different when
set in display style.  (See next section).

\subsection{Display Equations}
A numbered display equation---one set off by vertical space from the
text and centered horizontally---is produced by the \textbf{equation}
environment. An unnumbered display equation is produced by the
\textbf{displaymath} environment.

Again, in either environment, you can use any of the symbols
and structures available in \LaTeX\@; this section will just
give a couple of examples of display equations in context.
First, consider the equation, shown as an inline equation above:
\begin{equation}
  \lim_{n\rightarrow \infty}x=0
\end{equation}
Notice how it is formatted somewhat differently in
the \textbf{displaymath}
environment.  Now, we'll enter an unnumbered equation:
\begin{displaymath}
  \sum_{i=0}^{\infty} x + 1
\end{displaymath}
and follow it with another numbered equation:
\begin{equation}
  \sum_{i=0}^{\infty}x_i=\int_{0}^{\pi+2} f
\end{equation}
just to demonstrate \LaTeX's able handling of numbering.

\section{Figures}

The ``\verb|figure|'' environment should be used for figures. One or more images can be placed within a figure. If your figure contains third-party material, you must clearly identify it as such, as shown in the example below.
\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{sample-franklin}
  \caption{1907 Franklin Model D roadster. Photograph by Harris \& Ewing, Inc. [Public domain], via Wikimedia Commons. (\url{https://goo.gl/VLCRBB}).}
  \Description{The 1907 Franklin Model D roadster.}
\end{figure}

Your figures should contain a caption which describes the figure to the reader. Figure captions go below the figure. Your figures should {\bf also} include a description suitable for screen readers, to assist the visually-challenged to better understand your work.

Figure captions are placed {\it below} the figure.

\subsection{The ``Teaser Figure''}

A ``teaser figure'' is an image, or set of images in one figure, that are placed after all author and affiliation information, and before the body of the article, spanning the page. If you wish to have such a figure in your article, place the command immediately before the \verb|\maketitle| command:
\begin{verbatim}
  \begin{teaserfigure}
    \includegraphics[width=\textwidth]{sampleteaser}
    \caption{figure caption}
    \Description{figure description}
  \end{teaserfigure}
\end{verbatim}

\section{Citations and Bibliographies}

The use of \BibTeX\ for the preparation and formatting of one's references is strongly recommended. Authors' names should be complete --- use full first names (``Donald E. Knuth'') not initials (``D. E. Knuth'') --- and the salient identifying features of a reference should be included: title, year, volume, number, pages, article DOI, etc. 

The bibliography is included in your source document with these two commands, placed just before the \verb|\end{document}| command:
\begin{verbatim}
  \bibliographystyle{ACM-Reference-Format}
  
\section*{Acknowledgment}
This research was funded in part by NSF grant CCF-1703734.





\bibliographystyle{IEEEtran}

\section{Introduction}
The artifacts developers generate as they work and coordinate with others have long offered an important window into developers' workflow, needs, and activities, offering an indirect means to observe developers through their committed code, issues, comments, social media posts, and other artifacts\cite{lakhani2004, Mamykina2011, singer2014Twitter, MacLeod2015}.

Recently, a new form of artifact has emerged: the screencast. Early use of screencasting was often intended to offer developers tutorial content, replacing traditional text-based documentation by explaining how to use development tools or new APIs \cite{ellmann2017find}\cite{MacLeod2015}.
More recently, developers have begun to live-stream their own real-time work on open source software\cite{Faas2018}. These videos illustrate developers' work in action using their preferred development environment while working on real tasks in familiar and unfamiliar code. These videos are not rehearsed and aim to shows a direct view of the moment-to-moment behavior of developers engaged in real software development work (Figure \ref{fig:example}).

By showing how developers work moment-to-moment,  these videos offer an essential resource for software engineering research and education \cite{Haaranen2017}. They enable direct observation of developers building, debugging, and testing software that would otherwise require conducting a field study. These videos may help to illustrate existing as well as potentially new software engineering theories, strategies, and best practices. To showcase how to use strategies for tasks such as debugging \cite{Irvin1989}, software engineering educators might make use of examples of developers at work in a real-world task rather than create artificial examples.

However, using public videos for research and education today is difficult. First, videos are scattered over the Internet. Developers have many options for hosting their videos such as YouTube and Twitch. As these are filled with millions of videos in other categories, finding programming videos is hard and reliant on the videos' titles. 
Second, to identify developers exhibiting a specific behavior in a context, activity, or strategy, one cannot search directly for such videos. Instead, it is necessary to watch videos at random, which are typically 1-6 hours long, and hope that the behavior of interest is exhibited by the developer.


\begin{figure}
    \includegraphics[ width=\linewidth, keepaspectratio, clip, trim= 0cm 0cm 0cm 10cm]{figures/example}
    \caption{A video which shows a developer debugging.}
    \label{fig:example}
    \end{figure}
Creating a central repository of programming videos for research and education offers a potential solution to these problems. Videos could be analyzed by the research community and annotated to reflect the behavior they contain, identifying specific contexts, techniques, issues, strategies, and theories that they illustrate. Researchers or educators might share specific lists of videos, and repository users could search for videos with specific characteristics.




\section{MOTIVATING EXAMPLES}

To survey some of the potential benefits we envision of a central, annotated repository of programming videos, we describe several examples of its use for software engineering research and education. We name our proposed repository \textit{observe.dev}.

Sara is a professor who is teaching an undergraduate software testing class. She is planning to introduce test-driven development (TDD) to her students. She has prepared some materials for the class to teach the theory behind TDD. She also has made a simple example where she shows TDD in practice. However, she wants to also show how TDD is used in large projects and the practices, strategies, and tools developers use while applying it. She opens up \textit{observe.dev} and searches for videos of developers using TDD. The page lists several extended videos, including annotations for each denoting the time at which developers use TDD. She watches 20 minutes of video depicting developers practicing TDD and then shares this video with her students.

Deema is a new software engineering Ph.D. student who is trying to learn about theories of how developers navigate through code. While reading explanations of several theories in research papers, she discovers an information forging theory (IFT) paper which describes how developers navigate code while debugging \cite{Lawrance2013IFT}. She feels that her understanding of this theory is abstract, and a concrete example of a real developer navigating code that showcases this theory would help her more firmly grasp the concept. She uses \textit{observe.dev} to search for instances of developers browsing code using IFT. She finds 3 hours of a video illustrating a developer debugging within a large software project, with instances of IFT in action denoted with an annotation. After watching several minutes of a developer navigating code, she feels more confident in her understanding of this theory.      

George is a new Ph.D. student investigating the limitations of current debugging tools for web development and looking for a specific direction. Instead of trying to better understand how developers work with today's tools by conducting a lab study and recruiting participants to use a debugger, he decides to use \textit{observe.dev}. He searches for videos involving debugging tools for JavaScript using \textit{observe.dev} and finds over 50 instances. Watching these videos, he observes a number of instances in which the developer faces a similar challenge with these debugging tools. He annotates these instances and publishes them in \textit{observe.dev} to share them with the software engineering research community. Writing up a short paper on this observation, he suggests the need for a new form of debugging tool to address this challenge. He includes a link to videos with the new annotation for this challenge he added in \textit{observe.dev}.
\section{Preliminary work and Conclusion}
We have taken several initial steps towards creating a central repository of programming videos. We have collected over 40 hours of public programming videos. Our initial goal is to explore the value of these videos for research by investigating their use in understanding how developers debug and the strategies developers use which enable them to debug more effectively. 

In order to create a central repository of programming videos that serves both software engineering education and research, there are several important open questions which must be addressed.  What infrastructure and workflow is needed to effectively support and manage the contributions from the software engineering community? How can the annotated videos be effectively offered and displayed to students and instructors to facilitate their use in software engineering education? In what ways, if any, can tool support or automation make the process of curating and annotating videos which illustrate behaviors easier,  as some have begun to explore \cite{Ponzanelli2016, Moslehi2018}? Finally, what are the ethical implications of using these public videos of developers?  
We hope that a public repository of programming videos will provide a valuable resource for the software engineering community.
\section{Template Overview}
As noted in the introduction, the ``\verb|acmart|'' document class can be used to prepare many different kinds of documentation --- a double-blind initial submission of a full-length technical paper, a two-page SIGGRAPH Emerging Technologies abstract, a ``camera-ready'' journal article, a SIGCHI Extended Abstract, and more --- all by selecting the appropriate {\it template style} and {\it template parameters}.

This document will explain the major features of the document class. For further information, the {\it \LaTeX\ User's Guide} is available from \url{https://www.acm.org/publications/proceedings-template}.

\subsection{Template Styles}

The primary parameter given to the ``\verb|acmart|'' document class is the {\it template style} which corresponds to the kind of publication or SIG publishing the work. This parameter is enclosed in square brackets and is a part of the {\verb|documentclass|} command:
\begin{verbatim}
  \documentclass[STYLE]{acmart}
\end{verbatim}

Journals use one of three template styles. All but three ACM journals use the {\verb|acmsmall|} template style:
\begin{itemize}
\item {\verb|acmsmall|}: The default journal template style.
\item {\verb|acmlarge|}: Used by JOCCH and TAP.
\item {\verb|acmtog|}: Used by TOG.
\end{itemize}

The majority of conference proceedings documentation will use the {\verb|acmconf|} template style.
\begin{itemize}
\item {\verb|acmconf|}: The default proceedings template style.
\item{\verb|sigchi|}: Used for SIGCHI conference articles.
\item{\verb|sigchi-a|}: Used for SIGCHI ``Extended Abstract'' articles.
\item{\verb|sigplan|}: Used for SIGPLAN conference articles.
\end{itemize}

\subsection{Template Parameters}

In addition to specifying the {\it template style} to be used in formatting your work, there are a number of {\it template parameters} which modify some part of the applied template style. A complete list of these parameters can be found in the {\it \LaTeX\ User's Guide.}

Frequently-used parameters, or combinations of parameters, include:
\begin{itemize}
\item {\verb|anonymous,review|}: Suitable for a ``double-blind'' conference submission. Anonymizes the work and includes line numbers. Use with the \verb|\acmSubmissionID| command to print the submission's unique ID on each page of the work.
\item{\verb|authorversion|}: Produces a version of the work suitable for posting by the author.
\item{\verb|screen|}: Produces colored hyperlinks.
\end{itemize}

This document uses the following string as the first command in the source file: \verb|\documentclass[sigconf,screen]{acmart}|.

\section{Modifications}

Modifying the template --- including but not limited to: adjusting margins, typeface sizes, line spacing, paragraph and list definitions, and the use of the \verb|\vspace| command to manually adjust the vertical spacing between elements of your work --- is not allowed.

{\bf Your document will be returned to you for revision if modifications are discovered.}

\section{Typefaces}

The ``\verb|acmart|'' document class requires the use of the ``Libertine'' typeface family. Your \TeX\ installation should include this set of packages. Please do not substitute other typefaces. The ``\verb|lmodern|'' and ``\verb|ltimes|'' packages should not be used, as they will override the built-in typeface families.

\section{Title Information}

The title of your work should use capital letters appropriately - \url{https://capitalizemytitle.com/} has useful rules for capitalization. Use the {\verb|title|} command to define the title of your work. If your work has a subtitle, define it with the {\verb|subtitle|} command.
Do not insert line breaks in your title.

If your title is lengthy, you must define a short version to be used in the page headers, to prevent overlapping text. The \verb|title| command has a ``short title'' parameter:
\begin{verbatim}
  \title[short title]{full title}
\end{verbatim}

\section{Authors and Affiliations}

Each author must be defined separately for accurate metadata identification. Multiple authors may share one affiliation. Authors' names should not be abbreviated; use full first names wherever possible. Include authors' e-mail addresses whenever possible.

Grouping authors' names or e-mail addresses, or providing an ``e-mail alias,'' as shown below, is not acceptable:
\begin{verbatim}
  \author{Brooke Aster, David Mehldau}
  \email{dave,judy,steve@university.edu}
  \email{firstname.lastname@phillips.org}
\end{verbatim}

The \verb|authornote| and \verb|authornotemark| commands allow a note to apply to multiple authors --- for example, if the first two authors of an article contributed equally to the work. 

If your author list is lengthy, you must define a shortened version of the list of authors to be used in the page headers, to prevent overlapping text. The following command should be placed just after the last \verb|\author{}| definition:
\begin{verbatim}
  \renewcommand{\shortauthors}{McCartney, et al.}
\end{verbatim}
Omitting this command will force the use of a concatenated list of all of the authors' names, which may result in overlapping text in the page headers.

The article template's documentation, available at  \url{https://www.acm.org/publications/proceedings-template}, has a complete explanation of these commands and tips for their effective use.

\section{Rights Information}

Authors of any work published by ACM will need to complete a rights form. Depending on the kind of work, and the rights management choice made by the author, this may be copyright transfer, permission, license, or an OA (open access) agreement.

Regardless of the rights management choice, the author will receive a copy of the completed rights form once it has been submitted. This form contains \LaTeX\ commands that must be copied into the source document. When the document source is compiled, these commands and their parameters add formatted text to several areas of the final document:
\begin{itemize}
\item the ``ACM Reference Format'' text on the first page.
\item the ``rights management'' text on the first page.
\item the conference information in the page header(s).
\end{itemize}

Rights information is unique to the work; if you are preparing several works for an event, make sure to use the correct set of commands with each of the works.

\section{CCS Concepts and User-Defined Keywords}

Two elements of the ``acmart'' document class provide powerful taxonomic tools for you to help readers find your work in an online search. 

The ACM Computing Classification System --- \url{https://www.acm.org/publications/class-2012} --- is a set of classifiers and concepts that describe the computing discipline. Authors can select entries from this classification system, via \url{https://dl.acm.org/ccs/ccs.cfm}, and generate the commands to be included in the \LaTeX\ source. 

User-defined keywords are a comma-separated list of words and phrases of the authors' choosing, providing a more flexible way of describing the research being presented.

CCS concepts and user-defined keywords are required for all short- and full-length articles, and optional for two-page abstracts. 

\section{Sectioning Commands}

Your work should use standard \LaTeX\ sectioning commands: \verb|section|, \verb|subsection|, \verb|subsubsection|, and \verb|paragraph|. They should be numbered; do not remove the numbering from the commands. 

Simulating a sectioning command by setting the first word or words of a paragraph in boldface or italicized text is {\bf not allowed.}

\section{Tables}

The ``\verb|acmart|'' document class includes the ``\verb|booktabs|'' package --- \url{https://ctan.org/pkg/booktabs} --- for preparing high-quality tables. 

Table captions are placed {\it above} the table.

Because tables cannot be split across pages, the best placement for them is typically the top of the page nearest their initial cite.  To ensure this proper ``floating'' placement of tables, use the environment \textbf{table} to enclose the table's contents and the table caption.  The contents of the table itself must go in the \textbf{tabular} environment, to be aligned properly in rows and columns, with the desired horizontal and vertical rules.  Again, detailed instructions on \textbf{tabular} material are found in the \textit{\LaTeX\ User's Guide}.

Immediately following this sentence is the point at which Table~\ref{tab:freq} is included in the input file; compare the placement of the table here with the table in the printed output of this document.

\begin{table}
  \caption{Frequency of Special Characters}
  \label{tab:freq}
  \begin{tabular}{ccl}
    \toprule
    Non-English or Math&Frequency&Comments\\
    \midrule
    \O & 1 in 1,000& For Swedish names\\
    $\pi$ & 1 in 5& Common in math\\
    \$ & 4 in 5 & Used in business\\
    $\Psi^2_1$ & 1 in 40,000& Unexplained usage\\
  \bottomrule
\end{tabular}
\end{table}

To set a wider table, which takes up the whole width of the page's live area, use the environment \textbf{table*} to enclose the table's contents and the table caption.  As with a single-column table, this wide table will ``float'' to a location deemed more desirable. Immediately following this sentence is the point at which Table~\ref{tab:commands} is included in the input file; again, it is instructive to compare the placement of the table here with the table in the printed output of this document.

\begin{table*}
  \caption{Some Typical Commands}
  \label{tab:commands}
  \begin{tabular}{ccl}
    \toprule
    Command &A Number & Comments\\
    \midrule
    \texttt{{\char'134}author} & 100& Author \\
    \texttt{{\char'134}table}& 300 & For tables\\
    \texttt{{\char'134}table*}& 400& For wider tables\\
    \bottomrule
  \end{tabular}
\end{table*}

\section{Math Equations}
You may want to display math equations in three distinct styles:
inline, numbered or non-numbered display.  Each of
the three are discussed in the next sections.

\subsection{Inline (In-text) Equations}
A formula that appears in the running text is called an
inline or in-text formula.  It is produced by the
\textbf{math} environment, which can be
invoked with the usual \texttt{{\char'134}begin\,\ldots{\char'134}end}
construction or with the short form \texttt{\$\,\ldots\$}. You
can use any of the symbols and structures,
from $\alpha$ to $\omega$, available in
\LaTeX~\cite{Lamport:LaTeX}; this section will simply show a
few examples of in-text equations in context. Notice how
this equation:
\begin{math}
  \lim_{n\rightarrow \infty}x=0
\end{math},
set here in in-line math style, looks slightly different when
set in display style.  (See next section).

\subsection{Display Equations}
A numbered display equation---one set off by vertical space from the
text and centered horizontally---is produced by the \textbf{equation}
environment. An unnumbered display equation is produced by the
\textbf{displaymath} environment.

Again, in either environment, you can use any of the symbols
and structures available in \LaTeX\@; this section will just
give a couple of examples of display equations in context.
First, consider the equation, shown as an inline equation above:
\begin{equation}
  \lim_{n\rightarrow \infty}x=0
\end{equation}
Notice how it is formatted somewhat differently in
the \textbf{displaymath}
environment.  Now, we'll enter an unnumbered equation:
\begin{displaymath}
  \sum_{i=0}^{\infty} x + 1
\end{displaymath}
and follow it with another numbered equation:
\begin{equation}
  \sum_{i=0}^{\infty}x_i=\int_{0}^{\pi+2} f
\end{equation}
just to demonstrate \LaTeX's able handling of numbering.

\section{Figures}

The ``\verb|figure|'' environment should be used for figures. One or more images can be placed within a figure. If your figure contains third-party material, you must clearly identify it as such, as shown in the example below.
\begin{figure}[h]
  \centering
  \includegraphics[width=\linewidth]{sample-franklin}
  \caption{1907 Franklin Model D roadster. Photograph by Harris \& Ewing, Inc. [Public domain], via Wikimedia Commons. (\url{https://goo.gl/VLCRBB}).}
  \Description{The 1907 Franklin Model D roadster.}
\end{figure}

Your figures should contain a caption which describes the figure to the reader. Figure captions go below the figure. Your figures should {\bf also} include a description suitable for screen readers, to assist the visually-challenged to better understand your work.

Figure captions are placed {\it below} the figure.

\subsection{The ``Teaser Figure''}

A ``teaser figure'' is an image, or set of images in one figure, that are placed after all author and affiliation information, and before the body of the article, spanning the page. If you wish to have such a figure in your article, place the command immediately before the \verb|\maketitle| command:
\begin{verbatim}
  \begin{teaserfigure}
    \includegraphics[width=\textwidth]{sampleteaser}
    \caption{figure caption}
    \Description{figure description}
  \end{teaserfigure}
\end{verbatim}

\section{Citations and Bibliographies}

The use of \BibTeX\ for the preparation and formatting of one's references is strongly recommended. Authors' names should be complete --- use full first names (``Donald E. Knuth'') not initials (``D. E. Knuth'') --- and the salient identifying features of a reference should be included: title, year, volume, number, pages, article DOI, etc. 

The bibliography is included in your source document with these two commands, placed just before the \verb|\end{document}| command:
\begin{verbatim}
  \bibliographystyle{ACM-Reference-Format}
  
\section*{Acknowledgment}
This research was funded in part by NSF grant CCF-1703734.





\bibliographystyle{IEEEtran}

\section{Introduction}
The artifacts developers generate as they work and coordinate with others have long offered an important window into developers' workflow, needs, and activities, offering an indirect means to observe developers through their committed code, issues, comments, social media posts, and other artifacts\cite{lakhani2004, Mamykina2011, singer2014Twitter, MacLeod2015}.

Recently, a new form of artifact has emerged: the screencast. Early use of screencasting was often intended to offer developers tutorial content, replacing traditional text-based documentation by explaining how to use development tools or new APIs \cite{ellmann2017find}\cite{MacLeod2015}.
More recently, developers have begun to live-stream their own real-time work on open source software\cite{Faas2018}. These videos illustrate developers' work in action using their preferred development environment while working on real tasks in familiar and unfamiliar code. These videos are not rehearsed and aim to shows a direct view of the moment-to-moment behavior of developers engaged in real software development work (Figure \ref{fig:example}).

By showing how developers work moment-to-moment,  these videos offer an essential resource for software engineering research and education \cite{Haaranen2017}. They enable direct observation of developers building, debugging, and testing software that would otherwise require conducting a field study. These videos may help to illustrate existing as well as potentially new software engineering theories, strategies, and best practices. To showcase how to use strategies for tasks such as debugging \cite{Irvin1989}, software engineering educators might make use of examples of developers at work in a real-world task rather than create artificial examples.

However, using public videos for research and education today is difficult. First, videos are scattered over the Internet. Developers have many options for hosting their videos such as YouTube and Twitch. As these are filled with millions of videos in other categories, finding programming videos is hard and reliant on the videos' titles. 
Second, to identify developers exhibiting a specific behavior in a context, activity, or strategy, one cannot search directly for such videos. Instead, it is necessary to watch videos at random, which are typically 1-6 hours long, and hope that the behavior of interest is exhibited by the developer.


\begin{figure}
    \includegraphics[ width=\linewidth, keepaspectratio, clip, trim= 0cm 0cm 0cm 10cm]{figures/example}
    \caption{A video which shows a developer debugging.}
    \label{fig:example}
    \end{figure}
Creating a central repository of programming videos for research and education offers a potential solution to these problems. Videos could be analyzed by the research community and annotated to reflect the behavior they contain, identifying specific contexts, techniques, issues, strategies, and theories that they illustrate. Researchers or educators might share specific lists of videos, and repository users could search for videos with specific characteristics.




\section{MOTIVATING EXAMPLES}

To survey some of the potential benefits we envision of a central, annotated repository of programming videos, we describe several examples of its use for software engineering research and education. We name our proposed repository \textit{observe.dev}.

Sara is a professor who is teaching an undergraduate software testing class. She is planning to introduce test-driven development (TDD) to her students. She has prepared some materials for the class to teach the theory behind TDD. She also has made a simple example where she shows TDD in practice. However, she wants to also show how TDD is used in large projects and the practices, strategies, and tools developers use while applying it. She opens up \textit{observe.dev} and searches for videos of developers using TDD. The page lists several extended videos, including annotations for each denoting the time at which developers use TDD. She watches 20 minutes of video depicting developers practicing TDD and then shares this video with her students.

Deema is a new software engineering Ph.D. student who is trying to learn about theories of how developers navigate through code. While reading explanations of several theories in research papers, she discovers an information forging theory (IFT) paper which describes how developers navigate code while debugging \cite{Lawrance2013IFT}. She feels that her understanding of this theory is abstract, and a concrete example of a real developer navigating code that showcases this theory would help her more firmly grasp the concept. She uses \textit{observe.dev} to search for instances of developers browsing code using IFT. She finds 3 hours of a video illustrating a developer debugging within a large software project, with instances of IFT in action denoted with an annotation. After watching several minutes of a developer navigating code, she feels more confident in her understanding of this theory.      

George is a new Ph.D. student investigating the limitations of current debugging tools for web development and looking for a specific direction. Instead of trying to better understand how developers work with today's tools by conducting a lab study and recruiting participants to use a debugger, he decides to use \textit{observe.dev}. He searches for videos involving debugging tools for JavaScript using \textit{observe.dev} and finds over 50 instances. Watching these videos, he observes a number of instances in which the developer faces a similar challenge with these debugging tools. He annotates these instances and publishes them in \textit{observe.dev} to share them with the software engineering research community. Writing up a short paper on this observation, he suggests the need for a new form of debugging tool to address this challenge. He includes a link to videos with the new annotation for this challenge he added in \textit{observe.dev}.
\section{Preliminary work and Conclusion}
We have taken several initial steps towards creating a central repository of programming videos. We have collected over 40 hours of public programming videos. Our initial goal is to explore the value of these videos for research by investigating their use in understanding how developers debug and the strategies developers use which enable them to debug more effectively. 

In order to create a central repository of programming videos that serves both software engineering education and research, there are several important open questions which must be addressed.  What infrastructure and workflow is needed to effectively support and manage the contributions from the software engineering community? How can the annotated videos be effectively offered and displayed to students and instructors to facilitate their use in software engineering education? In what ways, if any, can tool support or automation make the process of curating and annotating videos which illustrate behaviors easier,  as some have begun to explore \cite{Ponzanelli2016, Moslehi2018}? Finally, what are the ethical implications of using these public videos of developers?  
We hope that a public repository of programming videos will provide a valuable resource for the software engineering community.