The Deep Program Understanding project aims to teach machines to understand complex algorithms, combining methods from the programming languages, software engineering and the machine learning communities.
Building “smart” software engineering tools requires learning to analyse and understand existing code and related artefacts such as documentation and online resources (e.g. StackOverflow). One of our primary concerns is the integration of standard static analysis methods with machine learning methods to create learning-based program analyses that can be used within software engineering tools. Such tools can then be used to find bugs, automatically retrieve or produce relevant documentation, or verify programs.
A core problem of machine learning is to learn algorithms that explain observed behaviour. This can take several forms, such as program synthesis from examples, in which an interpretable program matching given input/output pairs has to be produced; or alternatively programming by demonstration, in which a system has to learn to mimic sequences of actions.
Structured data such as programs represent a challenge for machine learning methods. The combination of domain constraints, known semantics and complex structure requires new machine learning methods and techniques. Our focus in this area is the analysis and generation of graphs, for which we have developed novel neural network architectures and generative procedures.
Below are some of the relevant publications of our group.
Chenglong Wang, Marc Brockschmidt, and Rishabh Singh. Pointing Out SQL Queries From Text. Tech Report, 2017.
Pratiksha Thaker, Daniel Tarlow, Marc Brockschmidt, and Alexander L. Gaunt. Semantics-aware Program Sampling. In DISCML @ NIPS’17.
Alexander L. Gaunt, Marc Brockschmidt, Nate Kushman, and Daniel Tarlow. Differentiable Programs with Neural Libraries. In ICML’17: International Conference on Machine Learning.
John K. Feser, Marc Brockschmidt, Alexander L. Gaunt, and Daniel Tarlow. Neural Functional Programming. In ICLR’17: International Conference on Learning Representations (Workshop Track).
Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. DeepCoder: Learning to Write Programs. In ICLR’17: International Conference on Learning Representations (Conference Track).
Chengtao Li, Daniel Tarlow, Alexander L. Gaunt, Marc Brockschmidt, Nate Kushman. Neural Program Lattices. In ICLR’17: International Conference on Learning Representations (Conference Track).
Alexander L. Gaunt, Marc Brockschmidt, Rishabh Singh, Nate Kushman, Pushmeet Kohli, Jonathan Taylor, and Daniel Tarlow. TerpreT: A Probabilistic Programming Language for Program Induction. Tech Report, 2016.
Marc Brockschmidt, Miltos Allamanis, Alexander L. Gaunt, Oleksandr Polozov. Generative Code Modeling with Graphs. In ICLR’19: International Conference on Learning Representations.
Pengcheng Yin, Graham Neubig, Miltos Allamanis, Marc Brockschmidt, Alexander L. Gaunt. Learning to Represent Edits. In ICLR’19: International Conference on Learning Representations.
Patrick Fernandes, Miltos Allamanis, Marc Brockschmidt. Structured Neural Summarization. In ICLR’19: International Conference on Learning Representations.
Miltos Allamanis, Marc Brockschmidt, and Mahmoud Khademi. Learning to Represent Programs with Graphs. In ICLR’18: International Conference on Learning Representations.
Miltos Allamanis, Earl T. Barr, Prem Devanbu, and Charles Sutton. A Survey of Machine Learning for Big Code and Naturalness. In ACM Computing Surveys 2018.
Santanu Dash, Miltos Allamanis, and Earl Barr. RefiNym: Using Names to Refine Types. In FSE’18: Foundations of Software Engineering.
Vincent Hellendoorn, Christian Bird, Earl Barr, Miltos Allamanis. Deep Learning Type Inference. In FSE’18: Foundations of Software Engineering.
Miltos Allamanis, and Marc Brockschmidt. SmartPaste: Learning to Adapt Source Code. Tech Report, 2017.
Mukund Raghothaman, Yi Wei, and Youssef Hamadi. SWIM: Synthesizing What I Mean. In ICSE’16: International Conference on Software Engineering.
Miltos Allamanis, Daniel Tarlow, Andrew D. Gordon, and Yi Wei. Bimodal Modelling of Source Code and Natural Language. In ICML’15: International Conference on Machine Learning.
Chris Maddison and Daniel Tarlow. Structured Generative Models of Natural Source Code. In ICML’14: International Conference on Machine Learning.
Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexander Gaunt, Raquel Urtasun, and Richard S. Zemel. Graph Partition Neural Networks for Semi-Supervised Classification. Tech Report, 2017.
Marc Brockschmidt, Yuxin Chen, Pushmeet Kohli, Siddharth Krishna, and Daniel Tarlow. Learning Shape Analysis. In SAS’17: Static Analysis Symposium.
Yujia Li, Richard Zemel, Marc Brockschmidt, and Daniel Tarlow. Gated Graph Sequence Neural Networks. In ICLR’16: International Conference on Learning Representations.
He Zhu, Aditya V. Nori, and Suresh Jagannathan. Learning Refinement Types. In ICFP ’15: International Conference on Functional Programming.
Marc Brockschmidt, Yuxin Chen, Byron Cook, Pushmeet Kohli, and Daniel Tarlow. Learning to Decipher the Heap for Program Verification. In Workshop on Constructive Machine Learning at ICML 2015. Best Paper Award.
He Zhu, Aditya V. Nori, and Suresh Jagannathan. Dependent Array Type Inference from Tests. In VMCAI ’15: Verification, Model Checking and Abstract Interpretation.
Varun Tulsian, Aditya Kanade, Rahul Kumar, Akash Lal, and Aditya V. Nori. MUX: Algorithm Selection for Software Model Checkers. In MSR ’14: Mining Software Repositories.
Rahul Sharma, Aditya V. Nori and Alex Aiken. Bias-Variance Tradeoffs in Program Analysis. In POPL ’14: Principles of Programming Languages.
Aditya V. Nori and Rahul Sharma. Termination Proofs from Tests. In ESEC-FSE ’13: Foundations of Software Engineering, August 2013.
Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken and Aditya V. Nori. Verification as Learning Geometric Concepts. In SAS ’13: Static Analysis Symposium.
Rahul Sharma, Saurabh Gupta, Bharath Hariharan, Alex Aiken, Percy Liang, and Aditya V. Nori. A Data Driven Approach for Algebraic Loop Invariants. In ESOP ’13: European Symposium on Programming.
Rahul Sharma, Aditya V. Nori, and Alex Aiken. Interpolants as Classifiers. In CAV ’12: Computer Aided Verification.
We have open-sourced many of our work and implementations.
dpu-utils: useful Python utilities for projects on deep program understanding.
gated-graph-neural-networks: A set of efficient TensorFlow implementations of graph neural networks that can handle large and sparse graphs.
constrained-graph-variational-autoencoders: code for constrained graph VAEs.
graph-partition-neural-network-samples: Sample code for Graph Partition Neural Networks.