The Development of Darwin's Origin of Species
Jaimie Murdock Colin Allen Simon DeDeo




The Development of Darwin's Origin of Species
    Jaimie Murdock, Colin Allen, and Simon DeDeo
    February 24, 2018
================================================










From 1837, when he returned to England aboard the HMS Beagle, to 1860, just after publication of The Origin of Species, Charles Darwin kept detailed notes of each book he read or wanted to  read. His notes and manuscripts provide information about decades of individual scientific practice. The textual nature of these records make them particularly amenable to computational analysis. 

Previously<cit.>, we located digitized versions of the English-language, non-fiction works listed as read in Darwin's reading notebooks. We trained topic models on the full texts of each reading, without using any information about his publications or additional historical metadata. We applied information-theoretic measures to the topic models of Darwin's readings to detect that changes in his reading patterns coincided with the boundaries of his three major intellectual projects in the period 1837-1860: a first shift in 1846 from his Beagle research journals to describing the Cirripedia — his “beloved barnacles’' — and a second shift in 1854 from barnacles to the work leading to The Origin of Species in 1859. While these intellectual phases are well-known to historians, the model offered a novel contribution by highlighting what changed in Darwin's reading habits: as he prepared his notes for the “large work on species,” his readings became more exploratory; i.e., as he was organizing his species notes his readings shifted subjects more often than when assembling his Beagle and Cirripedia notes. 


In this new work we apply the reading model to  five additional documents, four of them by Darwin: the first edition of The Origin of Species, two private essays stating intermediate forms of his theory in 1842 and 1844, a third essay of disputed dating, either preceding or succeeding the 1842 essay, and Alfred Russel Wallace's essay, which Darwin received in 1858. After describing the models and what they can show about the relationship between Darwin's writings and his readings over the course of his investigations, we address three historical inquiries, previously treated qualitatively:



  * the mythology of “Darwin's Delay,” that despite completing an extensive draft in 1844, Darwin waited until 1859 to publish The Origin of Species due to external pressures  <cit.>; 

  * the relationship between Darwin and Wallace's contemporaneous theories, especially in light of their joint presentation <cit.>; and

  * the dating of the “Outline and Draft” which was rediscovered in 1975 and postulated first as an 1839 draft preceding the Sketch of 1842 <cit.>, then as an interstitial draft between the 1842 and 1844 essays <cit.>.




§ QUERY SAMPLING THE WRITINGS


Our starting point for answering these questions was the previously trained topic model of his readings, as we were interested in how the readings influenced the writings. Topic models represent each text as a blend of different topics, with each topic being a probability distribution over the words in the collection. The models are statistically derived from a set of texts through joint inference of their word-topic and topic-document distributions <cit.>.  Query sampling allows a mixture of topics from a prior model to be assigned to documents not in the original training set. An initially random assignment of the words to the topics is revised iteratively until the assignment stabilizes using the same method used to train the original model. Because of the random starting point, running the query sampling process multiple times leads to different topic distributions for the same text.

This variability in outputs is something to be understood and harnessed, not feared, supporting different perspectives on the text.<cit.> For any text, there is not claimed to be a single “correct” interpretation but rather a set of interpretations in dialog with one another. Digital methods can augment existing debates in the humanities by providing different ways of looking at the text.  

We  approach the diversity of the sampled results by applying a clustering algorithm to the topic distributions, using the silhouette method to choose the number of clusters<cit.>. For The Origin, this method detects eight clusters, shown in Figure <ref>. Each cluster has a different highest-probability topic. These dominant topics characterize the primary interpretation of the text for each cluster. Inspection of the topics reveals that they are immediately applicable to The Origin. For example, pigeons (T49) provide a significant example for Darwin. The dominant topic of the largest cluster (T84) captures some key theoretic concerns with the words `'`development”, “creation”, “geological”, “organic”.




Because the topics fit to The Origin by query sampling are derived from the model of the readings, some of the words that have a high probability for a topic in the readings are likely not to appear in The Origin at all. For example, the second most likely word in T84, “moral” does not appear in the first edition of The Origin. Likewise, some of the geographic terms prominent in T177 do not appear in the book.  Indeed, T177 (with terms related to forests and South Asian geography and culture) presents an idiosyncratic view of The Origin. The statistical “perplexity” of this cluster with respect to the text confirms a relatively poor fit. Nonetheless, the assignment of T177 is grounded both in Darwin's reading of Falconer's Report on the teak forests of the Tenasserim provinces in 1853, and in his writing–Falconer is mentioned six times in the first edition of The Origin, and related issues are discussed in passages such as this, from chapter 5: 






[W]e have evidence, in the case of some few plants, of their becoming, to a certain extent, naturally habituated to different temperatures, or becoming acclimatised: ... trees growing at different heights on the Himalaya, were found in this country to possess different constitutional powers of resisting cold. Mr. Thwaites informs me that he has observed similar facts in Ceylon.

T177, like other clusters featuring geographical and ethnographic terms (T61, T135, T163), highlights how Darwin's own travels, correspondence with other travelers, and reading their published accounts expanded the global range of his evidence.












§ MEASURING COGNITIVE SURPRISE


We compared the writings to the readings and each other using an information-theoretic measure of cognitive surprise – Kullback-Leibler (KL) divergence<cit.> –  used in our previous study and which has proven successful in various cognitive science applications<cit.>.  Applied to the topic distributions derived by query sampling, KL divergence measures the extent to which the distribution of topics encountered in a new text violate the expectations based on the topic distributions in previously encountered texts.

KL divergence is an asymmetric measure, meaning that encountering B after A may generate a different amount of surprise than encountering A after B. Asymmetric measures are useful in many contexts: for example, travel time may be the more useful measure if it will take longer to climb a mountain than to go down it, even though the distance traveled in kilometers is the same. When a symmetric measure of distance between volumes is more appropriate, we use the symmetrical Jensen-Shannon distance (JSD), which is derived from the KL divergence and satisfies the mathematical properties of a distance metric.<cit.>




§ FINDINGS



 §.§ Explaining Darwin's Delay

Darwin began drafting his theory long before he started organizing his notes in 1854. With two private essays written in 1842 and 1844, it is a historical curiosity that he would wait until 1859 to publish his work, especially as immediately after finishing the second essay he wrote to his wife, Emma, with an addendum to his will concerning publication instructions should he die before finishing his work <cit.>. This period has become known as “Darwin's Delay” <cit.>. Theories about its cause include general fear of persecution <cit.>, the anonymous 1844 publication of Vestiges of the Natural History of Creation<cit.> highlighting gaps in Darwin's argument<cit.>, and extended illness <cit.>. We provide evidence for another motivation for the delay that has been proposed by others<cit.>: Darwin simply needed more time to gather evidence and develop his argument.

We use KL-divergence to trace the increase in cognitive surprise through Darwin's written presentations of his theory. Figure <ref> shows that with respect to the set of readings at any given time, The Origin is significantly more divergent than either of the earlier essays, and that the 1844 essay is slightly more divergent from the readings than the 1842 version. Interestingly, however, the 1842 and 1844 essays are more divergent from Darwin's readings at their respective times of writing, than the Origin is by 1859. This computational evidence supports the claim that Darwin's continued reading during the period between 1844 and 1859 was materially relevant to what he eventually wrote. 







 §.§ The rush to publish: Wallace's essay

Regardless of the primary cause of Darwin's delay, his sudden rush to publication is often attributed to the co-discovery of natural selection by  Wallace, whose own essay “On the tendency of varieties to depart infinitely from the original type” was co-published with an excerpt of Darwin's 1844 essay on 30 June 1858. When Darwin received Wallace's manuscript on 18 June 1858, Darwin had already been organizing his notes for The Origin for four years. Writing to Lyell, Darwin remarked on the impressive similarity to his earlier work:

I never saw a more striking coincidence. If Wallace had my M.S. sketch written out in 1842 he could not have made a better short abstract! <cit.>

We take Darwin's remark as both praising Wallace's work and emphasizing how much further his own ideas had developed by 1858. Darwin's observation indicated not just similarity between their work, but a specific similarity to his 1842 description of natural selection. The JSD measure[We use the symmetric JSD rather than KL-divergence, as JSD does not make any assumption about ordering of texts.] partially captures Darwin's observation: Wallace's work is more similar to the 1842 and 1844 essays than to The Origin. However, it is marginally—just over 1/100th of a bit—closer to the 1844 essay than the 1842 essay by this measure (top of Figure <ref>). Darwin's mention of his 1842 sketch may be interpreted as a generic reference to the earlier period, or it may reflect features of the 1842 sketch not accessible via topic modeling.




 §.§ Dating the “Outline and Draft”


Finally, we look at a manuscript originally discovered with the 1842 essay at the Darwin residence in 1896, but which was not included in The Foundations of the Origin of Species in 1909<cit.>. It had fallen into archival obscurity at the Cambridge University Libraries until  rediscovered in 1975 by Peter Vorzimmer,  who dated the outline to July 1839<cit.>.

However, scholarly consensus gravitated to a theory that the paper was an interstitial draft between the 1842 and 1844 essays <cit.>, based upon annotations on the manuscript itself and the reuse of headings from the draft in the 1844 essay which were not present in the 1842 sketch. Comparing it using JSD, we find that the draft is  further from The Origin than either the 1842 sketch or 1844 essay  (bottom of Figure <ref>). Moreover, the 1842 essay is further from The Origin than the 1844 essay. This finding provides some new evidence supporting Vorzimmer’s 1839 dating, although further investigation is necessary.








































































































