# Abstract

The retina is a complex nervous system which encodes visual stimuli before higher order processing occurs in the visual cortex. In this study we evaluated whether information about the stimuli received by the retina can be retrieved from the firing rate distribution of Retinal Ganglion Cells (RGCs), exploiting High-Density 64x64 MEA technology. To this end, we modeled the RGC population activity using mean-covariance Restricted Boltzmann Machines, latent variable models capable of learning the joint distribution of a set of continuous observed random variables and a set of binary unobserved random units. The idea was to figure out if binary latent states encode the regularities associated to different visual stimuli, as modes in the joint distribution. We measured the goodness of mcRBM encoding by calculating the Mutual Information between the latent states and the stimuli shown to the retina. Results show that binary states can encode the regularities associated to different stimuli, using both gratings and natural scenes as stimuli. We also discovered that hidden variables encode interesting properties of retinal activity, interpreted as population receptive fields. We further investigated the ability of the model to learn different modes in population activity by comparing results associated to a retina in normal conditions and after pharmacologically blocking GABA receptors ($GABA_C$ at first, and then also $GABA_A$ and $GABA_B$). As expected, Mutual Information tends to decrease if we pharmacologically block receptors. We finally stress that the computational method described in this work could potentially be applied to any kind of neural data obtained through MEA technology, though different techniques should be applied to interpret the results.

***This is an early version of the paper, a further revision is ongoing and this version will be updated when finalized.***

# Introduction

Retinal Ganglion Cells play a key role in the visual system, being the only connection between the retina and the areas of the brain dedicated to the process of the visual information. For this reason, understanding how the visual information is stored in the RGC neural activity is a very interesting neuroscientific problem. Multi-Electrode Array (MEA) Technology represents a novel and unique opportunity to study neural tissues, but defining novel techniques to make sense of the high dimensional output (spike trains) is an active problem. In this work we show that information related to the visual stimuli shown to the retina can be recovered from the RGC activity by modeling the population distribution with a probabilistic graphical model, through a novel approach that scales very well both in terms of the number of neurons recorded and in time. We also address the problem of extracting information related to population activities, with the same approach, showing that our model can learn latent representations of neural signals, such as population receptive fields.
Different works addressed the problem of decoding RGC activity. Bialek et al. first introduced the problem of decoding the activity of a movement-sensitive neuron of the fly visual system. Warland et al. showed that stimuli (in their case, photopic, spatially uniform, temporally broadband flicker) can be decoded from spike trains, and understood that decoding performance highly improved considering many spike trains. Pillow et al. evaluated a decoding approach based on correlations among different neurons. Marre et al. defined a linear decoder trained over a population of RGCs stimulated with a moving bar, and were able to decode the position of the bar from neural activity with high accuracy. Botella-Soler et al. addressed the problem of reconstructing the stimulus proposed to RGCs from spike activities, reporting promising results. A different area of research is related to infer the latent structure of RGCs. For example, Linderman et al. showed that type of cells and receptive field center locations can be unsupervisely learnt from spike trains, using a Generalized Linear Model . Our goal was different from the ones proposed in literature, since we were trying to unsupervisely extract information about the stimuli proposed, not building a decoder to reconstruct them. When we discuss about *latent representations*, it must not be confused with latent strutures discussed in , since the latent representations introduced in this work represent modes in the firing rate distribution.
To carry out the experiments, we acquired different datasets, obtained through High Density $64\times64$ MEA Technology, coupled with a projector showing stimuli to the retina on the chip. For our analysis we are thus considering activities of hundreds of neurons. In this work, we are making the assumption that the visual information is encoded in the spiking rate of the RGCs, following the rate coding approach. This choice was led by the fact we believe modeling the spike rate is more informative than modelling spike co-occurrence. We inferred the firing rates from the spike trains using log-Gaussian Cox processes . This choice was led by the fact that these models deal with uncertainty at different levels of abstraction, and so they well apt to the stochasticity of the neural activity. At this point, we needed a model able to find the regularities that - we hypothesize - are related to the visual stimuli. To this end, we chose to model the joint distribution of the firing rates with mean-covariance Restricted Boltzmann Mahchines, latent variable models capable of learning the joint distribution of a set of continuous observed random variables and a set of binary unobserved random variables (often referred to as latent variables). Once the model is trained, this bi-partition allows to represent regularities in the observed inputs through the inferred value of the latent binary variables. As a measure of the goodness of the modeling, we calculated the mutual information between the binary states and the different stimuli. First, we performed a two-hour experiment where square gratings with 800 µm-wide bars moving in 8 directions (0° to 360°) were used. After having validated the assumption that the regularities encoded by the different states are associated to different stimuli, we ran a test to evaluate the physiological plausibility of our findings. The experiment consisted in recording the RGC activity of an adult mouse retina (P45) when stimulated with three different types of moving square gratings (0.011, 0.023 and 0.045 cycle/degree), both in normal conditions and blocking $GABA_A$ and $GABA_B$ receptors. As expected, Mutual Information decreased after having blocked the receptors, because pharmacologically induced impairment led to to an unreliable population code. Finally, we evaluated whether natural stimuli can be retrieved from the firing rate distribution, using neural data from another experiment. The paper is organized as follows: in the following section the Methods are described, namely the different protocols used and the mathematical tools; the Results are reported in the next section; the last section includes the Conclusion, where also future work is detailed.

# Materials and Methods

## Data Acquisition

In this section, the different stimulation protocol and experimental modalities defined to validate our approach are detailed. All experiments were carried out with High Density $64 \times 64$ MEA Technology. Channels were arrenged on a $64 \times 64$ lattice and separated by $42 um$. Sampling rates were in the range of $7-8 kHz$, stimuli were shown to the retina with a frequency of $30 Hz$. The detailed description of the system is reported in . Figure 1 shows an overview of the experimental setup.

### Heterogeneous Grating Stimulation

We stimulated a P38 stage mouse's retina with an heterogeneous stimulation protocol. The idea was to evaluate if specific features shared by different images are encoded in specific retinal activation modes. Specifically, a moving square grating with 800 µm-wide bars was presented to the retina in 8 different orientations: 0°, 45°, 90°, 135°, 180°, 225°, 270° and 315°. For each orientation, ten repetitions of the stimuli were carried out. Figure 2 (*left*) shows different samples of stimuli.

### GABA Blockage

We recorded the activity of an adult mouse retina (P45 stage) when stimulated with three different moving square gratings in three different recording conditions. At first, the retina in normal conditions was subjected to the three following stimuli:

1.  Square grating at 0.011 cycles/degree

2.  Square grating at 0.023 cycles/degree

3.  Square grating at 0.045 cycles/degree

All the stimulation protocols had temporal frequency of $1 Hz$, Michelson contrast 0.5 and mean luminance $1.36 cd/m^2$. The following stages of the experiment consisted in repeating the same stimulations under two different pharmacological treatments aimed at progressively blocking $GABA$ receptors and hence impairing the functional behaviour of the inhibitory circuitry. Specifically, $TPMPA$ $150 µM$ was added first to block $GABA_C$ receptors and Bicuculline $10 µM$ was successively added to also block $GABA_A$ and $GABA_B$ receptors. The rationale behind the experiment was recording the activity of a retina having a progressively more impaired inhibitory circuitry and checking whether the decreased spatio-temporal precision of the encoding expected from physiology \[ref?\] was captured by the mc-RBM.

### Natural scenes

Our last eperiment was defined to understand whether we can retrieve the regularities associated to more complex stimuli, such as natural scenes. To this end, a pilot study was performed by projecting on the retina a sequence of frames generated by scanning natural images following a smooth closed trajectory. Figure 2 (*right*) shows an example where the photo of a brick wall was used to extract the stimulation sequence. The idea behind this stimulation protocol was trying to simulate a smooth scanning movement over a natural scene and see how the retina reacted to it.

## Mean-Covariance Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are latent variable models (two-layer networks) capable of learning the joint distribution of a set of observed random variables and a set of binary unobserved random variables (often referred to as latent variables), in an unsupervised fashion. In this model, the input variables are connected with stochastic, binary feature detectors, through symmetric weights. The technical report by Hinton gives a detailed explaination of this model, in this section we report only the details necessary to a basic understanding of it. RBMs are energy models, and the energy associated to the joint configuration of visible and hidden units, in case of binary input and hidden units, is defined by: $$E(\textbf{v},\textbf{h}) = - \textbf{a}^T\textbf{v} - \textbf{b}^T\textbf{h} - \textbf{v}^T\textbf{W}\textbf{h}$$ where **v**,**h** are the values associated to visible and hidden units, respectively; **a**,**b** are the biases associated to those units and **W** is the weight matrix. The energy is related to the probability density of visible and hidden units via $$P(\textbf{v},\textbf{h}) \propto e^{-E(\textbf{v},\textbf{h})}$$ In case we want to model real-valued inputs, visible units are replaced with linear units with independent Gaussian noise, and the energy function becomes $$E(\textbf{v},\textbf{h}) = - \frac{1}{2}(\textbf{v} - \textbf{a})^T(\textbf{v} - \textbf{a}) - \textbf{c}^T\textbf{h} - \textbf{v}^T\textbf{W}\textbf{h}$$ assuming unitary standard deviation of the Gaussian noise associated to visible units. Mean-covariance RBMs are a modified version of the standard model where the hidden units are divided in two different sets: mean units and precision units. The energy function is thus divided in two different terms: $E_{mc}= E_m + E_c$, where the former is defined by (3) and the latter is defined by $$E_c(\textbf{v},\textbf{h}_c) = -\textbf{d}^T\textbf{h}_c - (\textbf{v}^T\textbf{R})^2\textbf{P}\textbf{h}_c$$ where **R** is the visible-factor weight matrix, **P** the factor hidden pooling matrix, and **d** the hidden bias vector. The conditional distribution of the hidden units given the visibles is given by $$P(\textbf{h}\arrowvert\textbf{v}) = \sigma\Big(\textbf{d} + \big((\textbf{v}^T\textbf{R}^2\textbf{P})\big)^T\Big)$$ The conditional distribution of the visible units given the hiddens is given by $$P(\textbf{v}\arrowvert\textbf{h}_c,\textbf{h}_m) \propto N(\Sigma\textbf{W}\textbf{h}_m,\Sigma)$$ where $$\Sigma = \big(\textbf{R}(diag\big(-\textbf{P}^T\textbf{h}_c)\big)\textbf{R}^T\big)^{-1}$$ We can unsupervisely train such energy-based models using a fast algorithm, *contrastive divergence* . For a detailed explaination of the learning procedure we recommend to read . Once the model is trained, this bi-partition allows to represent regularities in the observed inputs through the inferred value of the latent binary variables. An important fact is that conditioning on the latent variables, the observed ones are approximately jointly Gaussian distributed . The mean and covariance of these Gaussians are defined by the specific values of the formers. This means that each of the possible binary vectors representing the latent variable values is associated to one mode of the joint distribution of the inputs. This aspect makes mc-RBMs a very good model for our purposes of finding the regularities associated to visual stimuli shown to the retina. One of these sets is used to model the mean value of the inputs, the other is thought for explicitly modelling the covariance between the observed variables. The presence of this second set allows to obtain a much better fit of the data distribution than what can be achieved with simpler models like Gaussian Restricted Boltzmann Machines.
In our framework, each single unit corresponds to a different neuron, and the values fed in input are firing rate values at different time steps. To train the model, we need to give in input a matrix where the rows are the $N_s$ values of firing rates and the columns are associated to the $N_n$ neurons recorded. Following the usual way of training RBMs, we shuffled the rows of our dataset and divided it in mini-batches of size $100$: during the training procedure, weights were updated after the model has passed through one mini-batch, averaging over the gradients calculated for each sample of the mini-batch.

## Model evaluation

It can be difficult to interpret what RBMs have learnt, because we lack of a direct measure that defines how the model is fitting our data. For this work, we relied on information theory techniques to evaluate how well models learnt, and defined a visualization technique inspired by Spike Trigger Average to understand *what* the models had learnt. Specifically, to understand whether the modes encoded by mc-RBMs are effectively associated to particular visual stimuli, we decided to exploit Mutual Information (MI). It provides a measure about how mutually dependent two continuous or discrete random variables are: in our case, we calculated it between the stimuli poposed to the retina and the binary states learnt by the mc-RBMs. MI for discrete random variables is defined by the following equation $$I(X;Y) = \sum_{y \in Y}\sum_{x \in X}p(x,y)log\Big(\frac{p(x,y)}{p(x)p(y)}\Big)$$ where, in our case, $x$ and $y$ are associated to different stimuli and different binary states. We could easily perform this step, because we knew from the experiment protocol the stimuli associated to every firing rate sample. MI provides a measure (a real value in the range $0-1$) which tells us how well the model has encoded the regularities associated with the visual stimuli. MI can give us a general idea about the learning outcome and can be really helpful to compare different models. However, it does not carry any information about *what* the model has learnt. To this end, we defined a technique to understand the meaning of the different binary states, in terms of visual stimuli encoded. Such method is closely related to STA, and was implemented as follows: (i) one binary state is selected; (ii) the firing rate samples associated to that particular state are retrieved; (iii) the stimuli associated to those samples are averaged. Following this procedure, we can understand whether the regularities encoded by the mc-RBM states are associated to different stimuli. This technique can also be applied to the single binary units of the model, selecting one binary variable and then following steps (ii) and (iii).

# Results

Results show that information related to visual stimuli shown to the retina can be retrieved from the joint distribution of RGC firing rates. We proved it using mc-RBMs applied to firing rates inferred through log-Gaussian Cox processes. In addition we show that, while the binary states of the machines are associated to regularities related to visual stimuli, the single binary units encode interesting firing rate activities, that we interpreted as population receptive fields. To train the models, we used a variation of the contrastive divergence algorithm, the *persistent contrastive divergence* . In order to model only neurons that carry information relative to the stimuli, we chose to use for our experiments only neurons with a time-varying spike-rate. To this end, we calculated the maximum and the minimum values of the derivative of the firing rate for each neuron, removing the cells when the difference between the maximum and the minimum was below a threshold (arbitrarialy set to $10^{-8}$). In this way, we are removing not only silent neurons, but also neurons with a static, banal firing rate, namely a firing rate that surely does not carry any information about the stimuli. To distinguish the different cells, we performed Spike Sorting before inferring the firing rates, using Plexon Offline Sorter with the option t-dist EM. The following section is organized as follows: in the first section the results obtained through the visualization techniques described in previous section are reported, using data acquired during the grating experiment; in the following section, the results associated to the GABA blockage experiment are detailed; in the last section, the results obtained from the experiment with natural scenes are detailed.

## Decoding visual stimuli from RGC firing rates

The results reported in this section were achieved using data acquired with the modalities previously described, with a retina (stage P38) stimulated with heterogeneous gratings. In order to manage the complexity of the task, the chip was divided in 16 non-overlapping regions ($16\times16$ electrodes each) and the recorded data were processed by 16 different mcRBMs (shown in Figure 1). This was nececessary due to the memory limitations of the GPU used for the analysis, but is also in line with what happens in the retina where local signal correlations play an important role in encoding the visual input. Totally, 3923 neurons were recorded, but only 1344 were used for the experiments, following the selection described. Figure 4 shows the results obtained: the images reported were obtained following the technique described. Each row is associated to neurons recorded from different patches of the chip. From top to bottom: t0 (114 neurons), t4 (132 neurons), t7 (61 neurons), t8 (104 neurons). The first two columns contain images associated with different *binary states*, the last two columns contain images associated with different *binary units*. The interpretation of the results reported in the first two columns is straightforward: latent states encode regularities in RGC joint distribution associated to different phases of the stimuli proposed to the retina. This is a key result in our study, since the main goal was to figure out if such information could be recovered from the population activity. As can be observed, averaged pictures are smoother than original stimuli: this is a positive aspect of the results, since we can expect the population activity associated to one stimulus to be more similar to the activities associated to comparable stimuli, rather than activities associated to very different stimuli. The consequence of this similarity is that firing rate samples associated to similar stimuli are encoded in the same binary states. We observed that increasing the size of the mc-RBM hidden state, averages were less smooth. This is an obvious consequence, since the machine can learn up to $2^N$ different states, where $N$ is the number of hidden units. If $N$ is big, the model can learn a number of regularities big enough that it does not need to aggregate different activities in the same state. The interpretation of the results reported the last two columns is more difficult. Our explanation is that hidden units encode population receptive fields. With this consideration, the interpretation of the results would be that while hidden *states* encode regularities associated to *particular stimuli* shown to the retina, hidden *variables* encode regularities associated to biological properties of the retina, such as population receptive fields. It would be interesting to evaluate if these receptive fields are approximation of the ones of Lateral Geniculate Nucleus cells, since each LGN cell receives inputs from different RGCs, and in this framework we are considering ensemble of cells.

## Effects of GABA blockage on the model

After having validated that states learnt by the mc-RBM are associated to modes in the RGC joint distribution closely related to the stimuli proposed, we were interested in evaluating how induced impairment in the inhibitory circuitry is reflected in the latent states. As expected, GABA blockage results in a lower MI between the states and the stimuli. Results are reported in Figure 5, which pictures the MI scores associated to the three cases described, with three different types of grating stimuli. Figure 6 shows the distribution of the stimuli associated to an arbitrarily chosen state. The distribution related to the state learnt *before* pharmacological treatment is associated to a precise range of stimuli, as expected from results shown in previous section; the distribution of stimuli associated to a state learnt *after* the treatment is highly corrupted, due to the fact that retinal behaviour is abnormal, which results in an unreliable population code.

## Natural scenes

Finally, we were interested to understand if RGCs produce a reliable response also when stimulated with natural scenes (specifically, a video showing frames of a wall). In case of a reliable response, we believed that our model could capture the modes in the distribution also in this scenario. Results proposed in this section were achieved using a P? stage mouse's retina, and the model was fed with the activities of neurons from t5. The findings reported in Figure 7 are consistent with the findings of the gratings experiment: latent states encode the regularities associated with the stimuli proposed. With this kind of stimuli, latent variables encode smoother versions of the stimuli.

# Conclusion

We proposed a new method to decode population activity and extract latent representations of the neuronal signals. Such method can potentially scale to thousands of neurons and to very long temporal sequences, using state-of-the-art GPUs. We evaluated this approach with different protocols: the first experiment, with heterogeneous gratings showed us that stimuli can be retrieved from RGC activities and interesting properties sush as population receptive fields can be detected; the pharmacological blockage of GABA receptors was a validation step; finally, the natural scene (wall) experiment was carried out to understand if more complex stimuli could be retrieved from the signals. Further investigations will be made in order to better understand the properties learnt by the binary variables, through a fine analysis of the type of cells active in the different activations. We are also interested in evaluating if we can retrieve temporal evolution of different stimuli from the joint distribution of RGCs. Some preliminary results are reported in .

# Acknowledgments

This research received financial support from the 7th Framework Programme for Research of the European Commission, under Grant agreement no. 600847: RENVISION project of the Future and Emerging Technologies (FET) programme Neuro-bio-inspired systems (NBIS) FET-Proactive Initiative.


abstract: Talented public speakers have thousands of hours of practice. One means of improving public speaking skills is practice through improvisation, e.g. presenting an improvised presentation using an unseen slide deck. We present Tedric, a novel system capable of generating coherent slide decks based on a single topic suggestion. It combines semantic word webs with text and image data sources to create an engaging slide deck with an overarching theme. We found that audience members perceived the quality of improvised presentations using these generated slide decks to be on par with presentations using human created slide decks for the *Improvised TED Talk* performance format. Tedric is thus a valuable new creative tool for improvisers to perform with, and for anyone looking to improve their presentation skills.
author: Thomas Winters
; Kory W. Mathewson
bibliography: winters.bib
title: Automatically Generating Engaging Presentation Slide Decks

# Introduction

Public speaking is difficult due to many psychosocial factors: 40% of surveyed adults feel anxious about speaking in front of an audience . Practice and experience can make significant improvements to speech anxiety levels . However, the overpracticing of a talk can make the delivery stiff and rehearsed. Improvisational theater has been shown to be effective in developing skills for complex social interactions . Giving improvised presentations is thus a useful exercise for enhancing presentation skills and overcoming public speaking anxiety. The exercise itself is already performed by improvisational theater and comedy groups all around the world in several variants, such as in the form of an *"Improvised TED Talk"* . This work presents the novel idea of automatically generating presentations suitable for such an improvisational speaking practice exercise, as well as for improvisational comedy performance.

One of the difficulties in the improvised presentation format is that the slide deck is created prior to performance, while in improvisational comedy, the audience provides suggestions to shape the show. The premade slide deck will thus not align with the suggestion, but instead just has *"curated random"* slides . Another difficulty is learning how to build constructive slide decks for another performer to present. Creating such a slide deck requires time and attention, which can be costly and does not scale well when doing a large quantity of performances. To solve these problems, we present Tedric (*Talk Exercise Designer using Random Inspirational Combinations*), a co-creative generative system that is capable of creating engaging slide decks for improvised presentations. It is designed to fulfill a specific task of an improviser for a particular format, being the design of an improvised TED-talk slide deck . In the improvised presentation format, one performer usually designs a slide deck for the other performer beforehand. Previous systems in this domain have experimented with fulfilling the presenter role, e.g. by writing text scripts, creating comedic performers, and all-round improvisational comedy agents . Tedric on the other hand focuses on the task of the other performer, who creates the slide deck for the speaker. The system also differs from previous work in slide deck generation in that rather than summarizing text into slides, or converting outlines to well-designed slides, it actively creates its own presentation story and chooses the content itself. This system is thus a good example of co-creation between humans and machines, as both have responsibility in the creative process of performing an improvised presentation. Thanks to the speed advantage Tedric has over its human counterpart, it can generate a slide deck based on a suggestion from a live audience on the spot. Another advantage Tedric has, is that it follows certain good design guidelines, something which individuals new to this format have struggled to learn quickly .

# Slide deck generator model

The Tedric system is composed of several linked generators, specified in a presentation schema. In this paper, we focus on the *Improvised TED talk* presentation schema, which is meant to give comedic suggestions to spark the presenter's creativity during the presentation. However, it is easy to create similar schemas for more serious presentations (e.g. an improvised *Pecha Kucha* ) by taking the subset of slide generators that do not use comedic content sources.

## Schemas

The templates and schemas technique is a commonly used approach for generating content using certain constraints . In this project, the presentation schema specifies a slide seed generator, a set of slide generators, slide generator weight functions and frequency limitations on slide generator types. The slide generators themselves also use a different kind of schema to generate their content. Such a slide generator schema specifies a set of content sources to fill placeholders in a specific PowerPoint slide template, resulting in a coherent slide.

## Slide Seed Generator

A seed generator produces a list of slide seeds based on the overarching main presentation topic, provided by the audience. The default seed generator achieves this by randomly walking over links in ConceptNet, a semantic network of related words . For example, it could find that *cat* is related to *rug* because cats can be found laying on rugs (Figure ). If no useful link is found for a certain slide, it backtracks and looks at the slide seed before the previous slide seed. The default seed generator constrains the first and last slide to be about the given presentation topic, as well as making this main topic return every three to six slides. It thus ensures that the given main topic is clearly present in the presentation, while still allowing some deviations to related topics.

## Slide Generator

Slide generators use a given seed to generate a logical, themed slide for the presentation slide deck. During slide deck generation, the presentation schema iteratively picks one of its slide generators using their weight functions. These weight functions calculate the weight of a slide generator based on the current position of the slide to be generated and on how often similar slide generators can occur in one presentation. For example, the weight of the *"Title Slide Generator"* peaks for the first slide, and has a weight of zero everywhere else. Similarly, the several *"About Me"* and *"History"* slide generators prefer being in one of the first slides after the title slide, and thus have higher weights when choosing a slide generator for these positions. Each slide generator also has several associated tags. These tags are used to limit the number of certain slide types occurring in the same slide deck. For example, at most one anecdote slide is allowed, and no more than 20% of the total number of slides can be about a quote. For every slide, the presentation schema thus first creates a list of slide generators that can still be used given the slide generators used for other slides. It then calculates the weight for all these slide generators by giving the slide number and the total number of slides as arguments to their weight functions. It then picks the slide generator using roulette wheel selection on the weights of these slide generators.

After picking a slide generator, this generator then creates a slide based on a given seed, using its slide generation schema. To achieve this, it passes the seed to all content sources specified in this template. For example, using the seed *"rug"*, the *"Full Screen Giphy Generator"* passes this seed to Giphy to find a related gif, and to a title content source to generate a slide title using this seed word (Figure ).

The system currently contains 26 slide generators. They can be grouped in several different main categories based on the type of slide they produce, which are listed below.

### Title Slide.

The title slide generator creates a slide with a generated title about the main topic, based on a large set of templates. The seed is often transformed to a related action by using WikiHow (see Section ). The slide also contains a subtitle with a generated presenter name (if none is provided) and a generated scientific sounding field (e.g. *Applied Cat Physics*) (Figure ).

### About Me Slide.

There are several different slide generators introducing the speaker. They all generate texts and images visualizing the accompanied text, and can be about a location, a country, a job, a book and/or a hobby related to the presenter (Figure ).

### History Slide.

The system contains several generators that make up history related to the given seed. The *"Historical Figure Quote"* picks an old picture (from Reddit), generates a name (using a context-free grammar), invents an achievement (using a WikiHow action and templates) and finds a related quote (using GoodReads) to create a fake historical figure related to the seed (Figure ). There are also other slide generators using vintage pictures with captions relating to time. These history slides are used for adding depth and narrative to the generated presentation topic.

### Full Screen Image Slide.

Breaking up the slideshow with a full screen image is a great way for providing freedom as well as for grabbing attention. There are several variants of large image slide generators employed in Tedric, such as using Google Images to create a descriptive slide, where the speaker has a lot of freedom and breathing space to talk. Another variant uses a large animated gif and a title pointing the attention to the gif.

### Statement Slide.

Since the slide decks are meant for *Improvised TED talks*, we want them to feel as if they provide the audience with a take away message. To achieve this, we created several templates for bold statements, call to actions and anecdote prompts (Figure ). The system also uses several online sources (such as InspiroBot and GoodReads) to provide inspirational quotes.

### Multiple Captioned Images Slide.

Some slide generators use multiple images on one slide, with randomly picked captions that belong together underneath (e.g. *"Expectation"* & *"Reality"*). We found that these slides work best if the last captioned image uses content source providing odd images, as they help the presenter with creating a strong punchline if done correctly.

### Chart Slide.

We implemented several ways of creating charts using generated data, as these usually give the presentation the perception of having more scientific foundation. One chart generator uses histograms and pie charts about yes/no questions. It generates a question about the slide seed using templates, and a funny third answer based on a large collection of textual templates (Figure ). The data generator prefers generating high values for the funny answer. Similarly, ConceptNet provides a good source of related locations that can be used in a pie chart or histogram (Figure ). Another type creates scatter plots by adding noise to basic mathematical functions (e.g. $y=x^2$ and $y=log(x)$). The labels of the axis are then found by using links in ConceptNet. A third type uses subreddits about data visualization for finding interesting related charts.

### Conclusion Slide.

Conclusion slides use a similar structure to the multiple captioned image slides. These slides usually start with a descriptive image about the main topic and have an odd image at the end, in order to finish the show with a punch.

## Content Sources

Content sources are used in slide generators to provide them with either texts or images. The content sources produce this content based either on a suggestion or at random. Slide generators pass the seeds they receive from the presentation schema down to all of their content sources. Each content source then decides whether or not to use this seed. For example, if the content source uses a search engine, it might fall back to picking content at random in case the search engine does not return any results. Some content sources (e.g. a context-free grammar for generating full names) might not support using the seed at all, and thus always return completely random content. Ignoring the seed and picking content at random every once in a while can have a positive effect as it might provide more variety in a presentation. This is especially true when the slide seed generator is not able to find a lot of meaningful related words, e.g. for an uncommon, niche word.

The system uses two types of content sources: image sources and text sources. These use search systems, context-free grammars, templates and combinations of the three. One advantage search systems have over pure context-free grammars and templates is that they are usually larger, contain more diverse content and are frequently updated. This gives rise to more surprises to both the creators as well as the users of the system, as new entries are usually constantly being added to these sources, generating different content every time it is used. Since such sources are harder to predict, it is important to identify the style and flavour of the elements in the search system or corpus in order to successfully use it in a slide generator schema. For example, stock photo websites usually give more neutral images, whereas certain subreddits might give funny or cute images. Sources of similar flavour can then be combined into composite content sources, to increase the variety a slide generator can produce. For example, the Google Images data source can be combined with the stock photo data source, as both provide neutral images. The slide generators then use these content sources in places where they need data in such a specific style and flavour.

### Image Sources

The system uses a large variation of image sources. In order to find descriptive pictures, it uses Google Images and stock photo websites. The system also uses images from Inspirobot[^1], a bot that generates nonsensical inspirational quotes, and frames them using a thought-provoking background. We also built our own data generation system for the analytical slide generators. This data source is used to insert random data into charts using labels created by text sources. Reddit[^2] is also a great source of images, as a lot of its subreddits usually contain images in a particular style or flavour. Images from Reddit are usually intrinsically interesting, as users tend to have good reasons for thinking the picture is worth posting on Reddit. The number of upvotes for a Reddit post generally gives a good indication for the quality of the image as well. We specifically use subreddits about odd, punchline images (e.g. *r/hmmm*, *r/wtfstockphotos*, *r/eyebleach*), gifs (e.g. *r/gifs*, *r/nonononoYES*), historic and vintage pictures (e.g. *r/OldSchoolCool*, *r/TheWayWeWere*, *r/ColorizedHistory*), charts and other data visualisations (e.g. *r/dataisbeautiful*, *r/funnycharts*), book covers and outside pictures. The Reddit gif source is combined with the Giphy[^3] source to create a composite gif data source.

### Text Sources

We employed several types of text sources in this project. The first type utilizes search engines to find texts related to a slide seed. As such, the system uses GoodReads[^4] as a source for related quotes, to be used on slides about historic people or for large statements. Another such search engine is WikiHow[^5], which we use to find human actions related to a certain seed, or related to other actions. This is achieved by searching on WikiHow for the given input and looking at the titles of the found articles. Considering WikiHow article titles follow structures such as *"How to ..."* or *"5 ways to ..."*, it is trivial to extract actions related to the input from these. For example, searching for *"cat"* on WikiHow finds articles such as *"How to Pet a Cat"*, which will lead to the action *"to pet a cat"*, which can then for example be inserted into a template (e.g. "`Why We All Need to {seed.wikihow_action.title}`") to form the talk title *"Why We All Need to Pet a Cat"*.

For creating controllable text content sources with high variation, we built our own text generation language on top of Tracery, a text generation language using context-free grammars . The extensions allows us to specify external variables with custom functions in the template texts. These custom functions can be used on the externally specified variables, and thus transform them into right forms, or chain other functions. They can also be used as a predicate (e.g. checking if the variable is a noun). The transformation functions are used to change the form of the variable (e.g. verb conjugation, pronouns or string casing) or finding related texts (e.g. related actions using WikiHow or related concepts using ConceptNet ). This extension is implemented by first making Tracery generate text which might have external variables and functions specified between curly brackets. After this generation, the system checks if there are missing external variables, failed predicate checks on these variables and if they are transformable using the specified functions. If all conditions are satisfied, the variable is filled in, transformed and inserted in the textual template. Otherwise, the generator keeps trying until all conditions are met. The extended generative grammar language allows us to specify a large number of generators, such as generators for historic people names, for bold and inspirational statements, talk (sub)titles, related sciences, slide title variations and punchline chart question answers.

### Tupled Sources

For specifying certain slide generators, we required tupled data sources, i.e. data sources that inspire each other, or that are generated at the same time. For example, when generating a job, we want to generate an image based on the generated job function. When generating slides with two captions, these captions should sometimes also relate to each other (e.g. *"good"* & *"bad"*, *"what I initially did"* & *"actual solution"*). For this, the system thus also allows the generation of content using tupled data source.

## Parallelisation

Since the slide decks are meant to be generated in real-time in front of an audience, we need the generator to finish within a reasonable time frame. To achieve this, the generator has the option of generating all slides at once in parallel. This decreases the influence of slow (e.g. online, rate-limited) data sources. However, naively generating all slides in parallel causes issues with the validity of the slide deck, as the generator has several constraints that must hold between generated slides, such as limited duplicate images and restrictions on maximum tag occurrences of the used slide generators. To solve this, Tedric generates the slide seeds and selects some slide generators. It then fires all the slide generators at once without giving them any knowledge about other slides, since there are no other slides yet. After all initial slides are generated, Tedric sequentially checks every slide, increasing its saved knowledge about the slides after viewing every next slide. If a slide is breaking any of the constraints, its slide number is stored for the next generation round. After viewing all slides, a new slide generation round is fired, but this time with knowledge from all valid, generated slides. This repeats itself until no new slide breaks any constraints. At the end of this process, the generated slides respect the constraints specified by the presentation schema similar to slides generated in a serial way.

## Code

The code, documentation and link to an online demo of Tedric is available on <https://korymath.github.io/talk-generator/>.

# Evaluation

We evaluated the system in several different ways. First, we made volunteers present improvised presentations and formally give feedback to all presentations. This feedback is then analysed to compare the quality of human created slide decks versus generated slide decks. Second, we evaluated the time performance of the full slide generation process in a context similar to the expected use case context, to evaluate if it completes within a reasonable time frame. Third, we informally evaluated the system by training some performers using generated slide decks, and making them perform the format on stage using slide decks generated during the performance.

## Quantitative results

To evaluate the quality of our system, we compared the perceived quality of generated slide decks with slide decks created by humans by performing them in the improvised presentation format with eight volunteers. In total, six talks were given. We generated three slide decks on the spot based on the audience suggestion for the presentation topic, and used three slide decks created by three different people that were used before to perform this format on stage . These six presentations were presented in mixed order, so that the participants did not realize that there were two different sets of presentation sources. Participants had to indicate whether they were presenting or listening to the presentation, and had to give two ratings, one for the given presentation as a whole, and one for the slide deck quality. We used a five-level scale, with five being the highest score (*Awful, Bad, Average, Good, Amazing*). A disadvantage of this evaluation is that the performative nature might be a confounding variable. However, this might also help the participants assess the quality of the slide deck better than rating the slide deck in separation.

With the gathered data[^6] on Figure , we can check if the source of the slide deck is correlated with the quality of the slide deck. When performing a t-test on the dataset to see if the distributions are different, we find $p=0.039$. This value allows us to reject, within the 5% confidence interval, the hypothesis that they have the same distribution. However, when performing a $\chi^2$ test on contingency table without scores of 1 (as none were present), then $p=0.148$, which is not significant enough to reject that they have a similar distribution. This is likely due to the fact that there were only 48 observations. Either way, the generated slides generally do not seem score lower in our sample than the human created slide decks.

## Qualitative Results

The feedback forms also had a section where the participants could fill in short, textual comments about their given slide decks, of which they did not know the source. There were 30 comments in total, which we described in a thematic analysis using codes that describe whether they are positive (POS) or negative (NEG) about the human created slide deck (H), generated slide deck (G), theme and presenter as well as common reasons for these sentiments (Figure ).

Some presenters had complained that it was quite unpredictable whether their given human-created slide decks would feature a new slide or an additional animation next, making it harder to present. Some participants also complained that these human-created slide deck had nothing to do with the talk itself, and that it often put too many words in the presenter's mouth. However, they did enjoy that slide decks had clever jokes, nice variation and backreferences with returning images.

Some participants complimented the generated slides on their nice structure and on the ease to link them to the presentation topic. The participants wanted to try a variation after seeing five talks, and proposed to test out the format in a panel show setting, with one moderator and two guests. The generated slide deck (*G3* in Figure ) did not support this type of talk well, as it was too focused on one person (e.g. in the *"About Me"* slide), giving rise to some negative comments about the content. The amount of text was too high, as there were three improvisers on stage, who all wanted to add new information as well. In order to support this type of presentation exercise well in the future, we will thus need an additional, revised version of the presentation schema.

## Time Performance

Since our generator is used in front of a live audience, it should be able generate within a reasonable time frame. To optimally estimate the timings of the typical use case of this generator, we mimicked such a situation as well as possible. Firstly, we used 500 of the most common English nouns[^7] as presentation topics for the generated slide decks, as these fairly accurately represent audience suggestions. Choosing more common nouns as topics are also a worst-case estimate for time performance. This is due to less common nouns having less relations, which increase the odds of slide topic generator cache hits, which also leads to more similar slide topics, increasing the probabilities of hitting content generator caches. Secondly, we ran the timing experiment on a normal laptop with download speeds fluctuating between 80-150 megabits per second, as this is similar to the set-up a goal user might have. Thirdly, the slide decks were not only represented in-memory, but also saved to a PowerPoint file. Fourthly, we chose to generate slide decks of seven slides, as this is a nice length used for practising the format, as well as for presenting snappy, humorous presentations. Increasing this length should not increase the generation time with a similar factor due to the fact that the slides are generated in parallel rounds.

We can see on Figure that 94.4% of slide decks are generated within a 20-second time window, with a median of 12.29 seconds, and a maximum of 38 seconds (and one outlier of 58 seconds). The time spent on actual computations in the program has a median of 2.49 seconds, meaning that about 80% of the total generation time is spent waiting on the response of remote content sources. Better internet connections can thus decrease the generation time even further. We can conclude from this experiment that the generation time of Tedric is low enough to be used in presentation workshops and on stage.

## Experiences on Exercise and On Stage Performance

We organized a small workshop and coached a group of four improvisational theater actors using this generator. They quickly improved their presenting methods. The narrative of their stories initially lacked logic and did not link back to the generated title of the talk, or convinced the audience about the importance of the topic. They also rarely dared using a bold statement right before going to the next slide, a technique for making the audience laugh (e.g. *"The next slide will provide the solution to this problem."*). These skills were easily trained using the generator, and at the end of a three hour workshop, all participants gave talks that avoided these mistakes. Later that week, they were put on stage during a comedy festival, where people could leave at any time to see other comedy acts. They managed to keep all thirty to forty people interested enough to stay seated during all four talks. Furthermore, they were able to make the audience laugh during almost every slide, thanks to the inspiring slide decks. We see this as a type of collaborative creativity between humans and computers. However, one thing we noticed was that the energy of the audience dropped when the fourth talk began. A likely reason for this has been described in the past as an *"ice cream after an ice cream situation"* . One way of overcoming this problem in the future would be to implement more types of presentation schemas, which could be achieved by slightly varying the current presentation schema (e.g. presentation schemas preferring more art, science or nature slides).

## Interpretation

In the evaluation sample, the generated slide decks score higher on average than the human-created ones, and make the presenters give better presentations overall (Figure ). One explanation for this is that the generated presentations are more related to the given audience suggestions, whereas the premade slide decks are not. This results in a more coherent presentation. Even though this is a small evaluation sample, it is a good indication that the system is performing well and is usable for enhancing the performance of presenters. We also showed that the generator is valuable and usable for coaching improvisers and even works well on stage.

# Future Work

There are several extensions we propose for future work. Obvious extensions are adding more variation to Tedric by adding more types of slide generators, different flavoured presentation schemas and slide deck themes. We hope that making this tool available for other performers might also incentivize more tech-savvy users to add their new slide generators to the code repository, giving rise to such additional variations.

This research also opens the path to using genetic algorithms for more human-curated co-creation of slides. This could be achieved by iteratively proposing several slides to the user, and giving the selected slide(s) higher fitness values. A slide cross-over operator could use the slide generator to swap the content filled in certain placeholders with content from another slide. Similarly, a mutation operator could just regenerate content in a certain position by calling this part of the generator again. A user would then be able to make slides evolve using human curation, while the system keeps its important role in the creative aspects. In this paper, we mainly put our focus on autonomous seeded slide generation, as the goal was to achieve real-time generation during performances.

One more ambitious goal of this research is to build an end-to-end system which can, in real-time, iteratively generate a slide deck using interaction-based machine learning models . This means that every next slide could be adapted to the response of the audience to previous slides, in order to further optimize engagement. For these types of sequence generative models, the training data set must contain many examples of rich, contextual, structured pairs. In order to generate such interactive presentations with e.g. neural models, we must develop an understanding of how to automatically generate a presentation comparable to a human using rules and heuristics, as we started in this paper. We hope that our tool is used by performers who see the advantages of our system, as such a community could help us build a data set to open the path to a more interactive slide deck generation system.

# Conclusion

In this paper, we presented a system for generating engaging slide decks called Tedric. This system can be used for both training presentation skills as well as for performing improvisational comedy in the *Improvised TED Talk* format.

There are several advantages our system has over humans having to create slide decks. Firstly, it has the advantage of being able to create slide decks about an overarching theme decided by an audience suggestion. We showed in our evaluation that this might increase the perceived quality of the given presentation. Secondly, it performs the task of creating slide decks for this format several orders of magnitude faster than humans can. Thirdly, it significantly lowers the barrier to perform improvised presentations. Not only is it able to follow good guidelines, something that performers often have struggled to learn quickly before, it also relieves the performers of the time consuming task to create slide decks for each other to practise.

The behaviour of the presented system is a type of co-creation between computers and humans, as it provides surprising slides to spark the creativity of the presenter, similar to how humans creating a slide deck would. We thus hope that this system will not only improve the presentation skills of humans, but also make them more acceptant towards computers aiding their creativity.

### Acknowledgments

Thank you to the reviewers for their time and attention reviewing this work, as well as for the insightful comments. Thanks to Julian Faid and Dr. Piotr Mirowski for advice and support in the creation of the software. Thank you to Lana Cuthbertson, the producer of TEDxRFT, and to the talented individuals at Rapid Fire Theatre for supporting innovative art. Thank you to all the performers who have done improvised TED talks and shared their views and opinions on how to structure and frame, your help in building the design guide was critical. Thanks to Shaun Farrugia for creating and hosting an online demo of Tedric, making it more easily available for performers of any background to use the system. Thank you to volunteers from the Belgian improvisational comedy group Preparee for volunteering for the evaluation.

[^1]: <http://inspirobot.me/>

[^2]: <https://reddit.com>

[^3]: <https://giphy.com/>

[^4]: <https://www.goodreads.com/>

[^5]: <https://wikihow.com>

[^6]: The evaluation data is made available on <https://github.com/korymath/tedric-analysis>

[^7]: <https://www.wordexample.com/list/most-common-nouns-english>