Classification of crystallization outcomes using deep convolutional neural networks
Andrew E. Bruno Patrick Charbonneau Janet Newman Edward H. Snell David R. So Vincent Vanhoucke Christopher J. Watkins Shawn Williams Julie Wilson



Center for Computational Research, University at Buffalo, Buffalo, New York, United States of America.

Department of Chemistry, Duke University, Durham,
	North Carolina, USA
Department of Physics, Duke University, Durham,
	North Carolina, USA

Collaborative Crystallisation Centre CSIRO, Parkville, Victoria, Australia.

Hauptman-Woodward Medical Research Institute and SUNY Buffalo, Department of Materials, Design, and Innovation, Buffalo, New York 14203, United States of America.

Google Brain, Google Inc., Mountain View, California, United States of America.

Google Brain, Google Inc., Mountain View, California, United States of America.
vanhoucke@google.com

IM&T Scientific Computing, CSIRO, Clayton South, Victoria, Australia

Platform Technology and Sciences, GlaxoSmithKline Inc., Collegeville, Pennsylvania, United States of America.

Department of Mathematics, University of York, York, United Kingdom.


The Machine Recognition of Crystallization Outcomes (MARCO) initiative has assembled roughly half a million annotated images of macromolecular crystallization experiments from various sources and setups. Here, state-of-the-art machine learning algorithms are trained and tested on different parts of this data set. We find that more than 94% of the test images can be correctly labeled, irrespective of their experimental origin. Because crystal recognition is key to high-density screening and the systematic analysis of crystallization experiments, this approach opens the door to both industrial and fundamental research applications.

Author summary:
Protein crystal growth experiments are routinely imaged, but the mass of accumulated data is difficult to manage and analyze. Using state-of-the-art machine learning algorithms on a large and diverse set of reference images, we manage to recapitulate the labels of a remarkably large fraction of the set. This automation should enable a number of industrial and fundamental applications.




Classification of crystallization outcomes using deep convolutional neural networks
    Julie Wilson
    December 30, 2023
===================================================================================





§ INTRODUCTION


X-ray crystallography provides the atomic structure of molecules and molecular complexes. These structures in turn provide insight into the molecular driving forces for small molecule binding, protein-protein interactions, supramolecular assembly and other biomolecular processes. The technique is thus foundational to molecular modeling and design. Beyond the obvious importance of structure information for understanding and altering the role of biomolecules, it also has important industrial applications. The pharmaceutical industry, for instance, uses structures to guide chemistry as part of a “predict first” strategy <cit.>, employing expert systems to reduce optimization cycle times and more effectively bring medicine to patients.  Yet, despite decades of methodological advances, crystallizing molecular targets of interest remains the bottleneck of the entire crystallography program in structural biology.

Even when crystallization is facile, it is microscopically rare; for macromolecules it is also uncommon <cit.>.  Experimental trials typically involve: (i) mixing a purified sample with chemical cocktails designed to promote molecular association, (ii) generating a supersaturated solution of the desired molecule via evaporation or equilibration, and (iii) visually monitoring the outcomes, before (iv) optimizing those conditions and analyzing the resultant crystal with an X-ray beam. One hopes for the formation of a crystal instead of non-specific (amorphous) precipitates or of nothing at all. In order to help run these trials, commercial crystallization screens have been developed; each screen generally contains 96 formulations designed to promote crystal growth.  Whether these screens are equally effective or not <cit.> remains debated, but their overall yield is in any case paltry.
Typically fewer than 5% of crystallization attempts produce useful results (with a success rate as low as 0.2% in some contexts <cit.>).

The practical solution to this hurdle has been to increase the convenience and number of crystallization trials.  To offset the expense of reagents and scientist time, labs routinely employ industrial robotic liquid handlers, nanoliter-size drops, and record trial outcomes using automated imaging systems <cit.>. Hoping to compensate for the rarity of crystallization, commercially available systems readily probe a large area of chemical space with minimal sample volume with a throughput of ∼1000 individual experiments per hour. 

While liquid handling is readily automated, crystal recognition is not.  Imaging systems may have made viewing results more comfortable than bending over a microscope, but crystallographers still manually inspect images and/or drops, looking for crystals or, more commonly, conditions that are likely to produce good crystals when optimized.  This human cost makes crystal recognition a key experimental bottleneck within the larger challenge of crystallizing biomolecules <cit.>. A typical experiment for a given sample includes four 96-well screens at two temperatures, i.e., 768 conditions (and can have up to twice that <cit.>). Assuming that it takes 2 seconds to manually scan a droplet (and noting that the scans have to be repeated, as crystallization is time dependent), simply looking at a single set of 96 trials over the lifetime of an experiment can take the better part of an hour [This estimate is based on personnal communication with five experienced crystallographers at GlaxoSmithKline: 2 seconds/observation × 8 observations × 96 wells. Note that current technology can  automatically store and image plates at about 3 min/plate.]. For the sake of illustration, the U.S. Structural Science group at GlaxoSmithKline performs ∼1200 96-well experiments per year.  If the targeted observation schedule were rigorously followed, the group would spend a quarter of the year staring at drops, of which the vast majority contains no crystal. Recording outcomes and analyzing the results of the 96 trials would further increase the time burden. Current operations are already straining existing resources, and the approach simply does not scale for proposed higher-density screening <cit.>. 
 
Crystal growth is also sufficiently uncommon that the tolerance for false negatives is almost nil. Yet most crystallographers are misguided in thinking that they themselves would never miss identifying a crystal given an image containing an crystal, or indeed miss a crystal in a droplet viewed directly under a microscope <cit.>. In fact, not only do crystallographers miss crystals due to lack of attention through boredom, they often disagree on the class an image should be assigned to. An overall agreement rate of ∼ 70% was found when the classes assigned to 1200 images by 16 crystallographers were compared <cit.>. (When considering only crystalline outcomes, agreement rose to ∼ 93%.) Consistency in visual scoring was also considered by Snell et al. when compiling a ∼ 150,000 image dataset <cit.>. They found that viewers give different scores to the same image on different occasions during the study, with the average agreement rate for scores on a control set at the beginning and middle of the study being 77%, rising to 84% for the agreement in scores between the middle and end of the study. Crystallographers also tend to be optimistically biased when scoring their own experiments <cit.>. A better use of expert time and attention would be to focus on scientific inquiry.  

An algorithm that could analyze images of drops, distinguish crystals from trivial outcomes, and reduce the effort spent cataloging failure, would present clear value both to the discipline and to industry.  Ideally, such an algorithm would act like an experienced crystallographer in:


  * recognizing macromolecular crystals appropriate for diffraction experiments;

  * recognizing outcomes that, while requiring optimization, would lead to crystals for diffraction experiments;

  * recognizing non-macromolecular crystals;

  * ignoring technical failures;

  * identifying non-crystalline outcomes that require follow up;

  * being agnostic as to the imaging platform used;

  * being indefatigable and unbiased;

  * occurring in a time frame that does not impede the process;

  * learning from experience.

Such an algorithm would further reduce the variance in the assessments, irrespective of its accuracy. A high-variance, manual process is not conducive to automating the quality control of the system end-to-end, including the imaging equipment. Enhanced reproducibility enables traceability of the outcomes, and paves the way for putting in place measurable, continuous improvement processes across the entire imaging chain.

Automated crystallization image classifications that attempt to meet the above criteria have been previously attempted. The research laboratories that first automated crystallization inspection quickly realized that image analysis would be a huge problem, and concomitantly developed algorithms to interpret them <cit.>.
None of these programs was ever widely adopted. This may have been due in part to their dependence on a particular imaging system, and to the relatively limited use of imaging systems at the time. Many of the early image analysis programs further required very time consuming collation of features and significant preprocessing, e.g., drop segmentation to locate the experimental droplet within the image. To the best of our knowledge, there was also no widespread effort to make a widely available image analysis package in the same way that that the diffraction oriented programs have been organized, e.g., the CCP4 package <cit.>.

Can a better algorithm be constructed and trained?  In order to help answer this question, the Machine Recognition of Crystallization Outcomes (MARCO) initiative was set up <cit.>. MARCO assembled a set of roughly half a million classified images of crystallization trials through an international collaboration with five separate institutions.  Here, we present a machine-learning based approach to categorize these images. Remarkably, the algorithm we employ manages to obtain an accuracy exceeding 94%, which is even above what was once thought possible for human categorization. This suggests that a deployment of this technology in a variety of laboratory settings is now conceivable. The rest of this paper is as follows.  Section <ref> describes the dataset and the scoring scheme, Sec. <ref> describes the machine-learning model and training procedure, Secs. <ref> and <ref> describe and discuss the results, respectively, and Sec. <ref> briefly concludes.





§ MATERIAL AND METHODS





 §.§ Image Data


The MARCO data set used in this study contains 493,214 scored images from five institutions (See Table <ref> <cit.>). The images were collected from imagers made from two different manufacturers (Rigaku Automation and Formulatrix), which have different optical systems, as well as by the in-house imaging equipment built at the Hauptman-Woodward Medical Research Institute (HWMRI) High-Throughput Crystallization Center (HTCC). Different versions of the setups were also used – some Rigaku images are collected with a true color camera, some are collected as greyscale images. The zoom extent varies, with some imagers set up to collect a field-of-view (FOV) of only the experimental droplet, and some set for the FOV to encompass a larger area of the experimental setup. The Rigaku and Formulatrix automation imaged vapor diffusion based experiments while the HTCC system imaged microbatch-under-oil experiments. A random selection of 50,284 test images was held out for validation. Images in the test set were not represented in the training set. The precise data split is available from the MARCO website <cit.>.
 


 §.§ Labeling

Images were scored by one or more crystallographers. As the dataset is composed of archival data, no common scoring system was imposed, nor were exemplar images distributed to the various contributors.  Instead, existing scores were collapsed into four comprehensive and fairly robust categories: clear, precipitate, crystal, and other.  This last category was originally used as a catchall for images not obviously falling into the three major classes, and came to assume a functional significance as the classification process was further investigated.  Examination of the least classifiable five percent of images indeed revealed many instances of process failure, such as dispensing errors or illumination problems. These uninterpretable images were then labelled as “other” during the rescoring, which added an element of quality control to the overall process <cit.>.



 §.§ Relabeling

After a first baseline system was trained (see Sec. <ref>), the 5% of the images that were most in disagreement with the classifier (independently of whether the image was in the training or the test set), were relabeled by one expert, in order to obtain a systematic eye on the most problematic images. 

Because no rules were established and no exemplars were circulated prior to the initial scoring, individual viewpoints varied on classifying certain outcomes. 
For example, the bottom 5% contained many instances of phase separation, where the protein forms oil droplets or an oily film that coats the bottom of the crystallization well.  Images were found to be inconsistently scored as “clear”, “precipitate”, or “other” depending on the amount and visibility of the oil film.  This example highlights the difficulty of scoring experimental outcomes beyond crystal identification. A more serious source of ambiguity arises from process failure.  Many of the problematic images did not capture experimental results at all.  They were out of focus, dark, overexposed, dropless, etc. Whatever labeling convention was initially followed, for the relabeling the “other” category was deemed to also diagnose problems with the imaging process.  

A total of 42.6% of annotations for the images that were revisited disagreed with the original label, suggesting somewhat high (1 to 2%) label noise in this difficult fraction of the dataset. For a fraction of this data, multiple raters were asked to label the images independently and had an inter-rater disagreement rate of approximately 22%. The inherent difficulty of assigning a label to a small fraction of the images is therefore consistent with the results of Ref. <cit.>. Table <ref> shows the final image counts after relabeling.





§ MACHINE LEARNING MODEL



The goal of the classifier here is to take an image as an input, and output the probability of it belonging to each of four classes (crystals, precipitate, clear, other) (see Fig. <ref>). The classifier used is a deep Convolutional Neural Network (CNN). CNNs, originally proposed in Ref. <cit.>, and their modern `deep' variants (see, e.g., Refs. <cit.> for recent reviews), have proven to consistently provide reliable results on a broad variety of visual recognition tasks, and are particularly amenable to addressing data-rich problems. They have been, for instance, state of the art on the very competitive ILSVRC image recognition challenge <cit.> since 2012.

This approach to visual perception has been making unprecedented inroads in areas such as medical imaging <cit.> and computational biology <cit.>, and have also shown to be human-competitive on a variety of specialized visual identification <cit.>. The chosen classifier is thus well suited for the current analysis.





 §.§ Model Architecture


The model is a variation on the widely-used Inception-v3 architecture <cit.>, which was state of the art on the ILSVRC challenge around 2015. Several more recent alternatives were tried, including Inception-ResNet-v2 <cit.>, and automatically generated variants of NASNet <cit.>, but none yielded any significant improvements. An extensive hyperparameter search was also conducted using Vizier <cit.>, also without providing significant improvement over the baseline.

The Inception-v3 architecture is a complex deep CNN architecture described in detail in Ref. <cit.> as well as the reference implementation <cit.>. We only describe here the modifications made to tailor the model to the task at hand.

Standard Inception-v3 operates on a 299x299 square image. Because the current problem involves very detailed, thin structures, it is plausible to assume that a larger input image may yield better outcomes. We use instead 599x599 images, and compress them down to 299x299 using an additional convolutional layer at the very bottom of the network, before the layer labeled  in the reference implementation. The additional convolutional layer has a depth (number of filters) of 16, a 3×3 receptive field (it operates on a 3×3 square patch convolved over the image) and a stride of 2 (it skips over every other location in the image to reduce the dimensionality of the feature map). This modification improved classification absolute accuracy by approximately 0.3%. A few other convolutional layers were shrunk compared to the standard Inception-v3 by capping their depth as described in Table <ref>, using the conventions from the reference implementation.



While these parameters are exhaustively reported here to ensure reproducibility of the results, their fine tuning is not essential to maximizing the success rate, and was mainly motivated by improving the speed of training. In the end, it was possible to train the model at larger batch size (64 instead of 32) and still fit within the memory of a NVidia K80 GPU (see more details in the training section below). Given the large number of examples available, all dropout <cit.> regularizers were removed from the model definition at no cost in performance.



 §.§ Data Preprocessing and Augmentation


The source data is partitioned randomly into 415990 training images and 47062 test images.

The training data is generated dynamically by taking random 599x599 patches of the input images, and subjecting them to a wide array of photometric distortions, identical to the reference implementation:


  * randomized brightness (± 32 out of 255),

  * randomized saturation (from 50% to 150%),

  * randomized hue (± 0.2 out of 0.5),

  * randomized contrast (from 50% to 150%).


In addition, images are randomly flipped left to right with 50% probability, and, in contrast to the usual practice for natural scenes which don't have a vertical symmetry, they are also flipped upside down with 50% probability. Because images in this dataset have full rotational invariance, one could also consider rotations beyond the mere 90^∘, 180^∘, 270^∘ that these flips provide, but we didn't attempt it here, as we surmise the incremental benefits would likely be minimal for the additional computational cost. This form of aggressive data augmentation greatly improves the robustness of image classifiers, and partly alleviates the need for large quantities of human labels.

For evaluation, no distortion is applied. The test images are center cropped and resized to 599x599.



 §.§ Training


The model is implemented in TensorFlow <cit.>, and trained using an asynchronous distributed training setup <cit.> across 50 NVidia K80 GPUs. The optimizer is RmsProp <cit.>, with a batch size of 64, a learning rate of 0.045, a momentum of 0.9, a decay of 0.9 and an epsilon of 0.1. The learning rate is decayed every two epochs by a factor of 0.94. Training completed after 1.7M steps (Fig. <ref>) in approximately 19 hours, having processed 100M images, which is the equivalent of 260 epochs. The model thus sees every training sample 260 times on average, with a different crop and set of distortions applied each time. The model used at test time is a running average of the training model over a short window to help stabilize the predictions.





§ RESULTS







 §.§ Classification


The original labeling gave rise to a model with 94.2% accuracy on the test set. Relabeling improved reported classification accuracy by approximately 0.3% absolute, with the caveat that the figures are not precisely comparable since some of the test labels changed in between.
The revised model thus achieves 94.5% accuracy on the test set for the four-way classification task. It overfits modestly to the training set, reaching just above 97% at the early-stopping mark of 1.7M steps. Table <ref> summarizes the confusions between classes. Although the classifier does not perform equally well on images from the various datasets, the standard deviation in performance from one set to another is fairly small, about 5% (see Table <ref>), compared to the overall performance of the classifier.






The classifier outputs a posterior probability for each class. By varying the acceptance threshold for a proposed classification, one can trade precision of the classification against recall. The receiver operating characteristic (ROC) curves can be seen in Fig. <ref>.


 §.§ Validation

At CSIRO C3 a workflow <cit.> has been set up which uses a variation of the analysis tool from DeepCrystal <cit.> to analyze newly collected crystallisation images and to assign either no score, `crystal’ score or `clear’ score. A total of 37,851 images were collected in Q1 2018 and assigned a human score by a C3 user were used as an independent dataset to test the MARCO tool. Within this dataset, 9746 images had been identified as containing crystals. The current, DeepCrystal tool (which assigns only ‘crystal’ or ‘clear’ scores) was found to have an overall accuracy rate of 74%, while the MARCO tool has 90%. Although this retrospective analysis doesn’t allow for a direct comparison of the ROC, the precision, recall and accuracy of the two tools all favor the MARCO tool, as shown in table 6. The precision achieved by MARCO on this dataset is also very similar to that seen for the CSIRO images in the training data.





 §.§ Pixel Attribution


We visually inspect to what parts of the image the classifier learns to attend by aggregating noisy gradients of the image with respect to its label on a per-pixel basis. The SmoothGrad <cit.> approach is used to visualize the focus of the classifier. The images in Fig. <ref> are constructed by overlaying a heat map of the classifier's attention over a grayscale version of the input image.



Note that saliency methods are imperfect and do not in general weigh faithfully all the evidence present in an image according to their contributions to the decision, especially when the evidence is highly correlated. Although these visualizations paint a simplified and very partial picture of the classifier's decision mechanisms, they help confirm that it is likely not picking up and overfitting to cues that are irrelevant to the task.



 §.§ Inference and Availability


The model is open-sourced and available online at <cit.>. It can be run locally using TensorFlow or TensorFlow Lite, or as a Google Cloud Machine Learning <cit.> endpoint. At time of writing, inference on a standard Cloud instance takes approximately 260ms end-to-end per standalone query. However, due to the very efficient parallelism properties of convolutional networks, latency per image can be dramatically cut down for batch requests.



§ DISCUSSION


Previous attempts at automating the analysis of crystallisation images  have employed various pattern recognition and machine learning techniques, including linear discriminant analysis <cit.>, decision trees and random forests <cit.>, and support vector machines <cit.>. Neural networks, including self-organizing maps, have also been used classify these images <cit.>, with the most recent involving deep learning <cit.>. However, all previous approaches have required a consistent set of images with the same field of view and resolution, in order to identify the crystallization droplet in the well <cit.>, and thereby restrict the analysis. Various statistical, geometric or textural features were then extracted, either directly from the image or from some transformation of the region of interest, to be used as variables in the classification algorithms. 

The results from various studies can be difficult to compare head-to-head because different groups present confusion matrices with the number of classes ranging from 2 to 11, only sometimes aggregating results for crystals/crytalline materials. There is also a tradeoff between the number of false negatives and the number of false positives. Yet most report classification rates for crystals around 80-85% even in more recent work <cit.>, in which missed crystals are reported with much lower rates. This advance comes at the expense of more false positives. For example, Pan et al. report just under 3% false negatives, but almost 38% false positives <cit.>.

As the trained algorithms are specific to a set of images, they are also restricted to a particular type of crystallisation experiment. Prior to the curation of the current dataset, the largest set of images (by far) came from the Hauptman-Woodward Medical Research Institute HTCC <cit.>. This dataset, which contains 147,456 images from 96 different proteins but is limited to experiments with the microbatch-under-oil technique, has been used in a number of studies <cit.>. 
Most notably, Yann et al. used a deep convolutional neural network that automatically extracted features, and reported a correct classification rates as high as 97% for crystals and 96% for non-crystals. Although impressive, these results were however obtained from a curated subset of 85,188 clean images, i.e., images with class labels on which several human experts agreed <cit.>. In order to validate our approach, we retrained our model to perform the same 10-way classification on that subset of the data alone without any tuning of the model's hyperparameters and achieved 94.7% accuracy, compared to the reported 90.8%.

In this context, the current results are especially remarkable. A crystallographer can classify images of experiments independently of the systems used to create those images. They can view an experiment with a microscope or look at a computer image and reach similar conclusions. They can look at a vapor diffusion experiment or a microbatch-under-oil setup and, again, asses either with confidence. Here, we show that this can be accomplished equally well, if not better, using deep CNNs. A benchtop researcher can classify many images, especially if they relate to a project that has been years in the making. For high-throughput approaches, however, that task becomes challenging. The strength of computational approaches is that each image is treated like the previous one, with no fatigue. Classification of 10,000 images is as consistent as classification of one. This advance opens the door for complete classification of all results in a high-throughput setting and for data mining of repositories of past image data.

Another remarkable aspect of our results is that they leverage a very generic computer vision architecture originally designed for a different classification problem – categorization of natural images – with very distinct characteristics. For instance, one can presume that the global geometric relationships between object parts would play a greater role in identifying a car or a dog in an image, compared to the very local, texture-like features involved in recognizing crystal-like structures. Yet no particular specialization of the model was required to adapt it to the widely differing visual appearances of the samples originating from different imagers. This convergence of approaches toward a unified perception architecture across a wide range of computer vision problems has been a common theme in recent years, further suggesting that the technology is now ready for wide adoption for any human-mediated visual recognition task.



§ CONCLUSION


In this work, we have collated biomolecular crystallization images for nearly half a million of experiments across a large range of conditions, and trained a CNN on the labels of these images. Remarkably, the resulting machine-learning scheme was able to recapitulate the labels of more than 94% of a test set. Such accuracy has rarely been obtained, and has no equal for an uncurated dataset. The analysis also identified a small subset of problematic images, which upon reconsideration revealed a high level of label discrepancy. This variability inherent to using human labeling highlights one of the main benefits of automatic scoring. Such accuracy also make conceivable high-density screening.

Enhancing the imaging capabilities by including UV or SONICC results, for instance, could certainly enrich the model. But several research avenues could also be pursued without additional laboratory equipment. In particular, it should be possible to leverage side information that is currently not being used.


  * The four-way classification scheme used is a distillation of 38 categories which are present in the source data. While these categories are presumed to be somewhat inconsistent across datasets, they could potentially provide an additional supervision signal.

  * Because one goal of this classifier is to be able to generalize across datasets, it would be worthwhile to investigate the contribution of techniques that have been designed to specifically reduce the effect of domain shift across data sources on the classification outcomes <cit.>.

  * Each crystallization experiment records a series of images taken over times. Using the timecourse information could enhance the success rate of the classifier <cit.>.


Note in closing that the current study focused on crystallization as an outcome, which is but a small fraction of the protein solubility diagram. Patterns of precipitation, phase separation, and clear drops, also provide information as to whether and where crystallization might occur. The success in identifying crystals, precipitate and clear can be thus also be used to accurately chart the crystallization regimes and to identify pathways for optimization <cit.>.  The application of this approach to large libraries of historical data may therefore reveal patterns that guide future crystallization strategies, including novel chemical screens and mutagenesis programs.



§ ACKNOWLEDGMENTS

We acknowledge discussions at various stages of this project with I. Altan, S. Bowman, R. Dorich, D. Fusco, E. Gualtieri, R. Judge, A. Narayanaswamy, J. Noah-Vanhoucke, P. Orth, M. Pokross, X. Qiu, P. F. Riley, V. Shanmugasundaram, B. Sherborne and F. von Delft.
PC acknowledges support from National Science Foundation Grant no. NSF DMR-1749374. 









64
fxundefined [1]
 ifx#1

fnum [1]
 #1firstoftwo
 secondoftwo
 
fx [1]
 #1firstoftwo
 secondoftwo
 





noop [0]secondoftwo

ref[1]@startlink#1@href
href[1]#1@endlink
anitize@url [0]`
12`$12`&12`#12`1̂2`_12`%12
startlink[1]
endlink[0]

rl [1]href #1












@bib@innerbibempty

[Harrison et al.(2017)Harrison, Lahue, Peng, Donofrio, Chang, and Glick]PredictFirst
  author author S Harrison, author B Lahue,
  author Z Peng, author
  A Donofrio, author C Chang,  and author M Glick, title title
  Extending ‘predict first’ to the design make-test cycle in
  small-molecule drug discovery, @noop  journal
  journal Future Med. Chem. volume 9,  pages 533–536 (year 2017)NoStop
[McPherson(1999)]mcpherson:1999
  author author Alexander  McPherson, @noop  title
  Crystallization of Biological Macromolecules (publisher
  CSHL Press, address Cold Spring Harbor, year
  1999) p. pages 586NoStop
[Chayen(2004)]chayen:2004
  author author Naomi E.  Chayen, title title
  Turning protein crystallisation from an art into a science, @noop
   journal journal Curr. Opin. Struct. Biol.  volume 14, pages 577–583 (year 2004)NoStop
[Fusco and Charbonneau(2016)]fusco:2016
  author author Diana Fusco and author Patrick Charbonneau, title title Soft matter
  perspective on protein crystal assembly, @noop  journal journal Colloids Surf. B: Biointerfaces volume 137, pages 22–31 (year
  2016)NoStop
[Ng et al.(2016)Ng,
  Dekker, Reardon, and von
  Delft]Ng:2016
  author author J.T. Ng, author C. Dekker, author P. Reardon,  and author F. von Delft, title
  title Lessons from ten years of crystallization
  experiments at the SGC, @noop  journal journal Acta Cryst. D volume 72, pages 224–235 (year 2016)NoStop
[Fazio et al.(2015)Fazio,
  Peat, and Newman]Fazio2015141
  author author V.J. Fazio, author T.S. Peat,   and author J. Newman,  title title Lessons for the future,  @noop  journal journal Methods Mol.
  Biol. volume 1261, pages 141–156
  (year 2015)NoStop
[Newman et al.(2012)Newman,
  Bolton, Muller-Dieckmann, Fazio, Gallagher, Lovell, Luft, Peat, Ratcliffe, Sayle, Snell, Taylor, Vallotton, Velanker, and von
  Delft]Newman:2012
  author author Janet Newman, author Evan E. Bolton, author Jochen Muller-Dieckmann, author Vincent J.  Fazio, author D. Travis  Gallagher, author David  Lovell, author Joseph R.  Luft, author Thomas S.  Peat, author David Ratcliffe, author Roger A. Sayle, author Edward H. Snell, author Kerry Taylor,
  author Pascal Vallotton,
  author Sameer Velanker,  and  author Frank von Delft,  title title On the need for an
  international effort to capture, share and use crystallization screening
  data, @noop  journal journal Acta
  Cryst. F volume 68, pages 253–258
  (year 2012)NoStop
[Kotseruba et al.(2012)Kotseruba, Cumbaa, and Jurisica]HTXIm_WWW
  author author Yulia Kotseruba, author Christian A Cumbaa,  and author Igor Jurisica, title title
  High-throughput protein crystallization on the world community grid and the
  gpu, @noop  journal journal J. Phys.
  Conf. Ser. volume 341, pages 012027
  (year 2012)NoStop
[Newman(2011)]NEWMAN201173
  author author Janet Newman, title title One plate, two
  plates, a thousand plates. how crystallisation changes with large numbers of
  samples, @noop  journal journal
  Methods volume 55, pages 73 – 80
  (year 2011)NoStop
[Zhang et al.(2017)Zhang,
  Gerard, Ikni, Ferry,
  Vuillard, Boutin, Ferte,
  Grossier, Candoni, and Veesler]ZHANG201718
  author author Shuheng Zhang, author Charline J.J. Gerard, author Aziza Ikni,
  author Gilles Ferry, author Laurent M. Vuillard, author Jean A. Boutin, author
  Nathalie Ferte, author
  Romain Grossier, author
  Nadine Candoni,  and author
  Stéphane Veesler, title
  title Microfluidic platform for optimization of
  crystallization conditions, @noop  journal journal J. Cryst. Growth volume 472, pages 18 – 28 (year 2017), note industrial
  Crystallization and Precipitation in France (CRISTAL-8), May 2016, Rouen
  (France)NoStop
[Thielmann et al.(2012)Thielmann, Koepke, and Michel]Thielmann201263
  author author Y. Thielmann, author J. Koepke,   and author H. Michel,  title title The esfri instruct core
  centre frankfurt: Automated high-throughput crystallization suited for
  membrane proteins and more, @noop  journal journal J. Struct. Funct. Genomics volume 13,  pages 63–69 (year 2012)NoStop
[Snell et al.(2008a)Snell, Lauricella, Potter, Luft, Gulde, Collins, Franks, Malkowski, Cumbaa, Jurisica, and  DeTitta]Snell:2008b
  author author Edward H.  Snell, author Angela M.  Lauricella, author Stephen A.  Potter, author Joseph R.  Luft, author Stacey M.  Gulde, author Robert J.  Collins, author Geoff  Franks, author Michael G.  Malkowski, author Christian  Cumbaa, author Igor Jurisica,  and author George T.  DeTitta, title title
  Establishing a training set through the visual analysis of crystallization
  trials. part II: Crystal examples, @noop  journal
  journal Acta Cryst. D volume 64,  pages 1131–1137 (year
  2008a)NoStop
[Note1()]Note1
  note This estimate is based on personnal communication with five
  experienced crystallographers at GlaxoSmithKline: 2 seconds/observation
  × 8 observations × 96 wells. Note that current technology can
  automatically store and image plates at about 3 min/plate.Stop
[Wilson()]Wilson:2006
  author author Julie Wilson, title title Automated
  classification of images from crystallisation experiments, in @noop
   booktitle Advances in Data Mining. Applications in
  Medicine, Web Mining, Marketing, Image and Signal Mining, editor edited by editor Petra Perner (publisher Springer Berlin Heidelberg) pp.  pages 459–473NoStop
[Snell et al.(2008b)Snell, Luft,
  Potter, Lauricella, Gulde,
  Malkowski, Koszelak-Rosenblum, Said, Smith, Veatch, Collins, Franks, Thayer, Cumbaa, Jurisica, and DeTitta]Snell:2008
  author author Edward H.  Snell, author Joseph R.  Luft, author Stephen A.  Potter, author Angela M.  Lauricella, author Stacey M.  Gulde, author Michael G.  Malkowski, author Mary  Koszelak-Rosenblum, author Meriem I. Said, author Jennifer L. Smith, author Christina K. Veatch, author Robert J. Collins, author Geoff Franks, author Max  Thayer, author Christian  Cumbaa, author Igor Jurisica,  and author George T.  DeTitta, title title
  Establishing a training set through the visual analysis of crystallization
  trials. part i: 1̃50 000 images, @noop  journal
  journal Acta Cryst. D volume 64,  pages 1123–1130 (year
  2008b)NoStop
[Hargreaves()]Hargreaves:2018 
  author author David Hargreaves, Personal communication@noop  NoStop
[Spraggon et al.(2002)Spraggon, Lesley, Kreusch, and  Priestle]Spraggon:2002
  author author Glen Spraggon, author Scott A. Lesley, author Andreas Kreusch,  and author John P. Priestle, title title Computational
  analysis of crystallization trials, @noop  journal
  journal Acta Cryst. D volume 58,  pages 1915–1923 (year 2002)NoStop
[Cumbaa and Jurisica(2005)]Cumbaa:2005
  author author Christian  Cumbaa and author Igor  Jurisica, title title
  Automatic classification and pattern discovery in high-throughput protein
  crystallization trials, @noop  journal journal J. Struct. Funct. Genomics volume 6,  pages 195–202 (year 2005)NoStop
[Kawabata et al.(2008)Kawabata, Saitoh, Takahashi, Asama, Mishima, Sugahara, and  Miyano]Kawabata:2008
  author author Kuniaki Kawabata, author Kanako Saitoh, author Mutsunori Takahashi, author Hajime Asama, author Taketoshi Mishima, author Mitsuaki Sugahara,  and author Masashi Miyano, title title Evaluation of
  protein crystallization state by sequential image classification,  @noop  journal journal Sensor Rev.  volume 28, pages 242–247 (year 2008)NoStop
[Buchala and Wilson(2008)]Buchala:2008
  author author Samarasena  Buchala and author Julie C.  Wilson, title title
  Improved classification of crystallization images using data fusion and
  multiple classifiers, @noop  journal journal Acta Cryst. D volume 64, pages 823–833 (year 2008)NoStop
[Winn et al.(2011)Winn,
  Ballard, Cowtan, Dodson,
  Emsley, Evans, Keegan,
  Krissinel, Leslie, McCoy,
  McNicholas, Murshudov, Pannu,
  Potterton, Powell, Read,
  Vagin, and Wilson]Winn:2011
  author author Martyn D.  Winn, author Charles C.  Ballard, author Kevin D.  Cowtan, author Eleanor J.  Dodson, author Paul Emsley, author Phil R. Evans,
  author Ronan M. Keegan, author Eugene B. Krissinel, author Andrew G. W. Leslie, author Airlie McCoy, author
  Stuart J. McNicholas, author
  Garib N. Murshudov, author
  Navraj S. Pannu, author
  Elizabeth A. Potterton, author
  Harold R. Powell, author
  Randy J. Read, author
  Alexei Vagin,  and author
  Keith S. Wilson, title
  title Overview of the ccp4 suite and current
  developments, @noop  journal journal
  Acta Cryst. D volume 67, pages
  235–242 (year 2011)NoStop
[Mar(2017)]Marco:2017
  https://marco.ccr.buffalo.edu title MAchine
  Recognition of Crystallization Outcomes (MARCO),  (year
  2017), note <https://marco.ccr.buffalo.edu> [Online;
  accessed 17-March-2018]NoStop
[Vallotton et al.(2010)Vallotton, Sun, Lovell, Fazio, and Newman]Vallotton:2010
  author author Pascal Vallotton, author Changming Sun, author David Lovell,
  author Vincent J. Fazio,  and  author Janet Newman, title title Droplit, an improved image analysis
  method for droplet identification in high-throughput crystallization
  trials, @noop  journal journal J.
  Appl. Crystallogr. volume 43, pages
  1548–1552 (year 2010)NoStop
[Rosa et al.()Rosa,
  Ristic, Marshall, and Newman]Rosa:2018
  author author N. Rosa, author M. Ristic,
  author B. Marshall,  and  author J. Newman, title title Keeping crystallographers app-y,  @noop  journal journal Acta Cryst. F  volume submittedNoStop
[Mele et al.(2014a)Mele, Li,
  Fazio, and Newman]Mele:2014
  author author Katarina  Mele, author Rongxin  Li, author Vincent J.  Fazio,  and author Janet  Newman, title title
  Quantifying the quality of the experiments used to grow protein crystals:
  the iqc suite, @noop  journal journal
  J. Appl. Cryst. volume 47, pages 1097–1106 (year 2014a)NoStop
[LeCun et al.(1989)LeCun,
  Boser, Denker, Henderson,
  Howard, Hubbard, and Jackel]lecun1989backpropagation
  author author Yann LeCun, author Bernhard Boser,
  author John S Denker, author Donnie Henderson, author Richard E Howard, author Wayne Hubbard,  and author Lawrence D Jackel, title title Backpropagation applied to handwritten
  zip code recognition, @noop  journal journal Neural computation volume 1, pages 541–551 (year 1989)NoStop
[LeCun et al.(2015)LeCun,
  Bengio, and Hinton]lecun2015deep
  author author Yann LeCun, author Yoshua Bengio,
   and author Geoffrey Hinton,  title title Deep learning,  @noop  journal journal Nature volume 521, pages 436 (year
  2015)NoStop
[Rawat and Wang(2017)]rawat2017deep
  author author Waseem Rawat and author Zenghui Wang, title title Deep convolutional
  neural networks for image classification: A comprehensive review,  @noop  journal journal Neural Comput.  volume 29, pages 2352–2449 (year 2017)NoStop
[Berg et al.(2010)Berg,
  Deng, and Fei-Fei]berg2010large
  author author A Berg, author J Deng,  and  author L Fei-Fei, 
  title
  Large scale visual recognition challenge (ILSVRC),  (year
  2010) note<http://www.image-net.org/challenges/LSVRC> NoStop
[Litjens et al.(2017)Litjens, Kooi, Bejnordi, Setio, Ciompi, Ghafoorian, van der Laak, van Ginneken, and Sánchez]litjens2017survey
  author author Geert Litjens, author Thijs Kooi,
  author Babak Ehteshami Bejnordi,
  author Arnaud Arindra Adiyoso Setio, author Francesco Ciompi, author Mohsen Ghafoorian, author Jeroen AWM van der Laak, author Bram van
  Ginneken,  and author Clara I Sánchez, title title A survey on
  deep learning in medical image analysis, @noop  journal journal Med. Image Anal. volume 42, pages 60–88 (year
  2017)NoStop
[Angermueller et al.(2016)Angermueller, Pärnamaa, Parts, and  Stegle]angermueller2016deep
  author author Christof  Angermueller, author Tanel  Pärnamaa, author Leopold  Parts,  and author Oliver  Stegle, title title
  Deep learning for computational biology, @noop  journal journal Mol. Syst. Biol. volume 12, pages 878 (year 2016)NoStop
[Krause et al.(2017)Krause,
  Gulshan, Rahimy, Karth,
  Widner, Corrado, Peng, and  Webster]krause2017grader
  author author Jonathan  Krause, author Varun  Gulshan, author Ehsan  Rahimy, author Peter  Karth, author Kasumi  Widner, author Greg S  Corrado, author Lily  Peng,  and author Dale R  Webster, title title
  Grader variability and the importance of reference standards for evaluating
  machine learning models for diabetic retinopathy, @noop 
  journal journal arXiv:1710.01711 [cs.CV]  volume (preprint) (year
  2017)NoStop
[Liu et al.(2017)Liu,
  Gadepalli, Norouzi, Dahl,
  Kohlberger, Boyko, Venugopalan, Timofeev, Nelson,
  Corrado et al.]liu2017detecting
  author author Yun Liu, author Krishna Gadepalli, author Mohammad Norouzi, author George E Dahl, author Timo Kohlberger,
  author Aleksey Boyko, author Subhashini Venugopalan, author Aleksei Timofeev, author Philip Q Nelson, author
  Greg S Corrado,  et al.,  title title Detecting cancer metastases
  on gigapixel pathology images, @noop  journal
  journal arXiv:1703.02442 [cs.CV] volume
  (preprint) (year 2017)NoStop
[Rumelhart et al.(1986)Rumelhart, Hinton, and Williams]rumelhart1986learning
  author author David E Rumelhart, author Geoffrey E Hinton,  and author Ronald J Williams, title title Learning
  representations by back-propagating errors, @noop  journal journal Nature volume 323,  pages 533 (year 1986)NoStop
[Bottou(2010)]bottou2010large
  author author Léon  Bottou, title title
  Large-scale machine learning with stochastic gradient descent, in  @noop  booktitle Proceedings of COMPSTAT'2010  (publisher Springer, year 2010) pp. pages 177–186NoStop
[Szegedy et al.(2016)Szegedy, Vanhoucke, Ioffe, Shlens, and Wojna]szegedy2016rethinking
  author author Christian  Szegedy, author Vincent  Vanhoucke, author Sergey  Ioffe, author Jon Shlens,  and author Zbigniew Wojna, title title Rethinking the
  inception architecture for computer vision, in @noop  booktitle Proceedings of the IEEE Conference on Computer Vision
  and Pattern Recognition (year 2016) pp. pages
  2818–2826NoStop
[Szegedy et al.(2017)Szegedy, Ioffe, Vanhoucke, and  Alemi]szegedy2017inception
  author author Christian  Szegedy, author Sergey  Ioffe, author Vincent  Vanhoucke,  and author Alexander A Alemi, title title Inception-v4, inception-resnet and the impact of residual
  connections on learning. @noop  journal journal arXiv:1602.07261 [cs.CV] volume
  (preprint) (year 2017)NoStop
[Zoph et al.(2017)Zoph,
  Vasudevan, Shlens, and Le]zoph2017learning
  author author Barret Zoph, author Vijay Vasudevan,
  author Jonathon Shlens,  and  author Quoc V Le, title title Learning transferable architectures for
  scalable image recognition, @noop  journal journal arXiv:1707.07012 [cs.CV] volume
  (preprint) (year 2017)NoStop
[Golovin et al.(2017)Golovin, Solnik, Moitra, Kochanski, Karro, and Sculley]golovin2017google
  author author Daniel Golovin, author Benjamin Solnik, author Subhodeep Moitra, author Greg Kochanski, author John Karro,
   and author D Sculley,  title title Google vizier: A service for
  black-box optimization, in @noop  booktitle
  Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge
  Discovery and Data Mining (organization ACM, year 2017) pp. pages 1487–1495NoStop
[Silberman and Guadarrama(2017)]slim
  author author Nathan Silberman and author Sergio Guadarrama, https://github.com/tensorflow/models/tree/master/research/slim title TensorFlow-Slim image classification model library, 
  (year 2017), note
  <https://github.com/tensorflow/models/tree/master/research/slim>
  [Online; accessed 02-February-2018]NoStop
[Srivastava et al.(2014)Srivastava, Hinton, Krizhevsky,
  Sutskever, and Salakhutdinov]srivastava2014dropout
  author author Nitish Srivastava, author Geoffrey Hinton, author Alex Krizhevsky, author Ilya Sutskever,  and author Ruslan Salakhutdinov, title title Dropout:
  A simple way to prevent neural networks from overfitting, @noop 
  journal journal J. Mach. Learn. Res. volume 15, pages 1929–1958 (year
  2014)NoStop
[Abadi et al.(2015)Abadi,
  Agarwal, Barham, Brevdo,
  Chen, Citro, Corrado,
  Davis, Dean, Devin,
  Ghemawat, Goodfellow, Harp,
  Irving, Isard, Jia,
  Jozefowicz, Kaiser, Kudlur,
  Levenberg, Mané, Monga,
  Moore, Murray, Olah,
  Schuster, Shlens, Steiner,
  Sutskever, Talwar, Tucker,
  Vanhoucke, Vasudevan, Viégas, Vinyals, Warden, Wattenberg, Wicke, Yu, and Zheng]tensorflow2015-whitepaper
  author author Martín  Abadi, author Ashish  Agarwal, author Paul  Barham, author Eugene  Brevdo, author Zhifeng  Chen, author Craig Citro, author Greg S. Corrado, author Andy Davis,
  author Jeffrey Dean, author Matthieu Devin, author
  Sanjay Ghemawat, author
  Ian Goodfellow, author
  Andrew Harp, author
  Geoffrey Irving, author
  Michael Isard, author
  Yangqing Jia, author
  Rafal Jozefowicz, author
  Lukasz Kaiser, author
  Manjunath Kudlur, author
  Josh Levenberg, author
  Dan Mané, author
  Rajat Monga, author
  Sherry Moore, author
  Derek Murray, author
  Chris Olah, author
  Mike Schuster, author
  Jonathon Shlens, author
  Benoit Steiner, author
  Ilya Sutskever, author
  Kunal Talwar, author
  Paul Tucker, author
  Vincent Vanhoucke, author
  Vijay Vasudevan, author
  Fernanda Viégas, author
  Oriol Vinyals, author
  Pete Warden, author
  Martin Wattenberg, author
  Martin Wicke, author
  Yuan Yu,  and author
  Xiaoqiang Zheng, https://www.tensorflow.org title TensorFlow:
  Large-scale machine learning on heterogeneous systems,  (year
  2015)NoStop
[Dean et al.(2012)Dean,
  Corrado, Monga, Chen,
  Devin, Mao, Senior,
  Tucker, Yang, Le et al.]dean2012large
  author author Jeffrey Dean, author Greg Corrado,
  author Rajat Monga, author Kai Chen, author
  Matthieu Devin, author
  Mark Mao, author Andrew Senior, author Paul  Tucker, author Ke Yang, author Quoc V Le,
  et al., title title Large scale
  distributed deep networks, in @noop  booktitle
  Advances in neural information processing systems (year
  2012) pp. pages 1223–1231NoStop
[Tieleman and Hinton(2012)]tieleman2012lecture
  author author Tijmen Tieleman and author Geoffrey Hinton, title title Lecture
  6.5-rmsprop: Divide the gradient by a running average of its recent
  magnitude, @noop  journal journal
  COURSERA: Neural networks for machine learning volume
  4, pages 26–31 (year 2012)NoStop
[Watkins(2018)]c3pipeline
  author author Chris Watkins, https://doi.org/10.4225/08/5a97375e6c0aa title C4, C3 Classifier Pipeline. v1. CSIRO. Software
  Collection.  (year 2018), note
  <https://doi.org/10.4225/08/5a97375e6c0aa> [Online; accessed
  09-May-2018]NoStop
[dee(2017)]deepcrystal
  http://www.deepcrystal.com title
  DeepCrystal,  (year 2017), note
  <http://www.deepcrystal.com> [Online; accessed 09-May-2018]NoStop
[Smilkov et al.(2017)Smilkov, Thorat, Kim, Viégas, and Wattenberg]smilkov2017smoothgrad
  author author Daniel Smilkov, author Nikhil Thorat, author Been Kim,
  author Fernanda Viégas,   and author Martin Wattenberg,  title title Smoothgrad: Removing noise
  by adding noise, @noop  journal journal
  arXiv:1706.03825 [cs.LG] volume (preprint)
  (year 2017)NoStop
[Vanhoucke(2018)]tfmodelsmarco
  author author Vincent Vanhoucke, http://github.com/tensorflow/models/tree/master/research/marco title Marco repository in TensorFlow Models,  (year 2018), note
  <http://github.com/tensorflow/models/tree/master/research/marco>
  [Online; accessed 01-May-2018]NoStop
[clo(2018)]cloudml
  https://cloud.google.com/ml-engine title
  Google Cloud Machine Learning Engine,  (year 2018),  note [Online; accessed 02-February-2018]NoStop
[Cumbaa et al.(2003)Cumbaa,
  Lauricella, Fehrman, Veatch,
  Collins, Luft, DeTitta, and  Jurisica]Cumbaa:2003
  author author Christian A.  Cumbaa, author Angela  Lauricella, author Nancy  Fehrman, author Christina  Veatch, author Robert  Collins, author Joe Luft, author George DeTitta,
   and author Igor Jurisica,  title title Automatic classification of
  sub-microlitre protein-crystallization trials in 1536-well plates,  @noop  journal journal Acta Cryst. D  volume 59, pages 1619–1627 (year 2003)NoStop
[Saitoh et al.(2005)Saitoh,
  Kawabata, Asama, Mishima,
  Sugahara, and Miyano]Saitoh:2005
  author author Kanako Saitoh, author Kuniaki Kawabata, author Hajime Asama, author Taketoshi Mishima, author Mitsuaki Sugahara,  and author Masashi Miyano, title title Evaluation of
  protein crystallization states based on texture information derived from
  greyscale images, @noop  journal journal Acta Cryst. D volume 61, pages 873–880 (year 2005)NoStop
[Bern et al.(2004)Bern,
  Goldberg, Stevens, and Kuhn]Bern:do5006
  author author Marshall  Bern, author David Goldberg, author Raymond C. Stevens,  and author Peter Kuhn, title title Automatic
  classification of protein crystallization images using a curve-tracking
  algorithm, @noop  journal journal J.
  Appl. Cryst. volume 37, pages
  279–287 (year 2004)NoStop
[Liu et al.(2008)Liu,
  Freund, and Spraggon]Liu20081187
  author author R. Liu, author Y. Freund,  and  author G. Spraggon, title title Image-based crystal detection: a
  machine-learning approach. @noop  journal journal Acta Cryst. D volume 64, pages 1187–95 (year 2008)NoStop
[Cumbaa and Jurisica(2010)]Cumbaa:2010
  author author Christian A.  Cumbaa and author Igor  Jurisica, title title
  Protein crystallization analysis on the world community grid, @noop
   journal journal J. Struct. Funct. Genomics  volume 11, pages 61–69 (year 2010)NoStop
[Pan et al.(2006)Pan,
  Shavit, Penas-Centeno, Xu,
  Shapiro, Ladner, Riskin,
  Hol, and Meldrum]Pan:2006
  author author Shen Pan, author Gidon Shavit,
  author Marta Penas-Centeno,
  author Dong-Hui Xu, author Linda Shapiro, author
  Richard Ladner, author
  Eve Riskin, author
  Wim Hol,  and author
  Deirdre Meldrum, title
  title Automated classification of protein
  crystallization images using support vector machines with scale-invariant
  texture and gabor features, @noop  journal journal Acta Cryst. D volume 62, pages 271–279 (year 2006)NoStop
[Po and Laine(2008)]Po20081926
  author author M.J. Po and author A.F. Laine,  title title Leveraging genetic algorithm
  and neural network in automated protein crystal recognition, in  @noop  booktitle Proceedings of the 30th Annual
  International Conference of the IEEE Engineering in Medicine and Biology
  Society, EMBS'08 - "Personalized Healthcare through Technology" (year 2008) pp. pages 1926–1929NoStop
[Yann and Tang(2016)]Yann:2016
  author author Margot Lisa-Jing  Yann and author Yichuan  Tang, title title
  Learning deep convolutional neural networks for x-ray protein
  crystallization image analysis, in @noop  booktitle Thirtieth AAAI Conference on Artificial Intelligence  (year 2016)NoStop
[Hung et al.()Hung,
  Collins, Weldetsion, Newland,
  Chiang, Guerrero, and Okada]Hung:2014
  author author Jeffrey Hung, author John Collins,
  author Mehari Weldetsion,
  author Oliver Newland, author Eric Chiang, author
  Steve Guerrero,  and author
  Kazunori Okada, title
  title Protein crystallization image classification
  with elastic net, in @noop  booktitle SPIE
  Medical Imaging, Vol. volume 9034 (publisher
  SPIE) p. pages 14NoStop
[Fusco et al.(2014)Fusco,
  Barnum, Bruno, Luft,
  Snell, Mukherjee, and Charbonneau]Fusco:2014
  author author Diana Fusco, author Timothy J. Barnum, author Andrew E. Bruno, author Joseph R. Luft,
  author Edward H. Snell, author Sayan Mukherjee,  and author Patrick Charbonneau, title title Statistical analysis of crystallization
  database links protein physico-chemical features with crystallization
  mechanisms, @noop  journal journal
  PLoS ONE volume 9, pages e101123
  (year 2014)NoStop
[Ganin and Lempitsky(2015)]ganin2015unsupervised
  author author Yaroslav  Ganin and author Victor  Lempitsky, title title
  Unsupervised domain adaptation by backpropagation, in @noop 
  booktitle International Conference on Machine
  Learning (year 2015) pp. pages
  1180–1189NoStop
[Bousmalis et al.(2016)Bousmalis, Trigeorgis, Silberman,
  Krishnan, and Erhan]bousmalis2016domain
  author author Konstantinos  Bousmalis, author George  Trigeorgis, author Nathan  Silberman, author Dilip  Krishnan,  and author Dumitru  Erhan, title title
  Domain separation networks, in @noop  booktitle Advances in Neural Information Processing Systems (year 2016) pp. pages 343–351NoStop
[Mele et al.(2014b)Mele, Lekamge,
  Fazio, and Newman]Mele:2013
  author author Katarina  Mele, author B. M. Thamali  Lekamge, author Vincent J.  Fazio,  and author Janet  Newman, title title
  Using time courses to enrich the information obtained from images of
  crystallization trials, @noop  journal journal Cryst. Growth Des volume 14, pages 261–269 (year 2014b)NoStop
[Snell et al.(2008c)Snell, Nagel,
  Wojtaszcyk, O'Neill, Wolfley, and Luft]Snell:2008c
  author author Edward H.  Snell, author Ray M.  Nagel, author Ann Wojtaszcyk, author Hugh O'Neill, author Jennifer L. Wolfley,  and author Joseph R. Luft, title title The application
  and use of chemical space mapping to interpret crystallization screening
  results, @noop  journal journal Acta
  Cryst. D volume 64, pages 1240–1249
  (year 2008c)NoStop
[Altan et al.(2016)Altan,
  Charbonneau, and Snell]Altan:2016
  author author Irem Altan, author Patrick Charbonneau,  and author Edward H.  Snell, title title
  Computational crystallization, @noop  journal
  journal Arch. Biochem. Biophys. volume
  602, pages 12–20 (year 2016)NoStop


