Summer school on Neurosymbolic Programming July 11-13 2022
Neurosymbolic programming is an exciting new area at the intersection of Program Synthesis and Machine Learning that aims to
learn models that incorporate program-like structure.
For this first summer school in Neurosymbolic Programming we aim to bring together students and researchers from different backgrounds who are interested
in learning about the state of the art, applications and open problems in the field.
The summer school will include lectures and hands-on tutorials from leading researchers in the area.
It will take place on July 11-13 at the California Institute of Technology in Pasadena, California USA.
Apply here to attend the summer school; the application deadline is May 13.
Participation in the summer school will be free of charge for accepted participants, but participants will be expected to cover their travel and lodging costs.
Limited funds for travel grants are available with priority given to graduate students.
All summer school participants will be expected to abide by the Summer School Code of Conduct
Monday July 11 2022 |
Tuesday July 12 2022 |
Wednesday July 13 2022 |
Monday July 11 2022 |
Time |
Event |
9:00 to 9:15 |
Day 1 Introductory remarks
|
9:15 to 10:00 |
Keynote 1: Jacob Andreas (MIT)
(info)
Learning to program by learning to read
In the age of deep networks, "learning" almost invariably means "learning from examples". Image classifiers are trained with large datasets of (labeled or unlabeled) images, machine translation systems with corpora of translated sentences, and robot policies with demonstrations. But when human learners acquire new concepts and skills, we often do so with richer supervision, especially in the form of language---we learn new concepts from exemplars accompanied by descriptions or definitions, and new skills from demonstrations accompanied by instructions. In natural language processing, recent years have seen a number of successful approaches to learning from task definitions and other forms of auxiliary language-based supervision. But these successes have been largely confined to tasks that also involve language as an input and an output. What will it take to make language-based training useful for other learning problems? In this talk, I'll present some recent results on using natural language to guide both search and library learning in inductive program synthesis, and discuss connections to the role of language in human concept learning.
|
10:30 to 10:45 |
Break |
10:45 to 12:00 |
Tutorial 1a: Yisong Yue (Caltech)/ Swarat Chaudhuri (UT Austin) / Jennifer Sun (Caltech)
(info)
Basics of Neurosymbolic Architectures
This part overviews the design of neurosymbolic architectures and their training. We will begin with an overview of conventional deep learning (i.e., purely neural architectures). Afterwards, we will introduce the concept of a domain specific language (DSL), which can include both symbolic and neural primitives. Any program or architecture can be designed using primitives from this DSL. We will then construct a few explicit neurosymbolic architectures and train their continuous parameters using gradient-based learning.
|
12:00 to 13:00 |
Lunch |
13:00 to 14:00 |
Talk: Guy Van Den Broeck (UCLA)
|
14:00 to 14:30 |
Break |
14:30 to 15:30 |
Tutorial 2a: Nadia Polikarpova (UCSD)/Shachar Itzhaky (Technion): Program Synthesis
(info)
Program Synthesis Tutorial Part 1
In this first part of the tutorial, students will get an introduction to symbolic program synthesis and will gain
some experience with existing synthesis tools.
|
15:30 to 15:45 |
Break |
15:45 to 16:45 |
Talk: Osbert Bastani (UPenn) |
16:45 to 18:00 |
TBA
|
18:00 to 20:00 |
Dinner |
Tuesday July 12 2022 |
Time |
Event |
9:00 to 9:15 |
Day 2 Introductory remarks
|
9:15 to 10:00 |
Keynote 2: Katie Bouman (Caltech)
|
10:30 to 10:45 |
BREAK |
10:45 to 12:00 |
Tutorial 1b: Yisong Yue (Caltech)/ Swarat Chaudhuri (UT Austin) / Jennifer Sun (Caltech)
(info)
Neurosymbolic Program Architecture Search
This part introduces two methods for automatically searching for the best program architecture within a DSL. The first method is a
basic enumerative (or exhaustive) search. The second is RobustFill. For both methods, we will
have hands-on exercises so that students can get a feel for how these methods work in practice. This part will conclude with a brief survey of other methods.
|
12:00 to 13:00 |
Lunch
|
13:00 to 14:00 |
Talk: Charles Sutton (Google)
|
14:00 to 14:15 |
Break |
14:15 to 15:15 |
Tutorial 3a: Vikash Mansinghka (MIT)
(info)
Probabilistic Programming Tutorial Part 1
|
15:15 to 15:30 |
Break |
15:30 to 16:30 |
Keynote 3: David Choi (DeepMind)
(info)
Competitive Programming with AlphaCode
Programming is a powerful problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent large-scale language models have demonstrated impressive abilities to generate code, however they still perform poorly on more complex tasks that require problem-solving skills, such as competitive programming problems. In this talk we'll present AlphaCode, the motivations of the project and the design decisions we made. AlphaCode is a system for code generation that achieved an average ranking of top 54.3% in simulated evaluations on popular, recent programming competitions on the Codeforces platform. AlphaCode's success stemmed from: large transformer-based models, using a novel combination of architectural, training, and prompting modifications; extensive datasets; efficient large-scale sampling; and filtering and clustering-based sample selection. This marks the first time an artificial intelligence system has performed competitively in programming competitions.
|
16:30 to 18:00 |
Outing
|
18:00 to 20:00 |
Dinner |
Wednesday July 13 |
Time |
Event |
9:00 to 9:15 |
Day 3 introductory remarks
|
9:15 to 10 |
Talk: Armando Solar-Lezama (MIT)
(info)
Understanding the world with Code
This talk will describe how neurosymbolic program synthesis can help understand the world.
|
10:30 to 10:45 |
Break |
10:45 to 12:00 |
Tutorial 1c: Yisong Yue (Caltech)/ Swarat Chaudhuri (UT Austin) / Jennifer Sun (Caltech)
(info)
Neurosymbolic Program Architecture Search (continued)
This part introduces two methods for program architecture search. The first method is on using Admissible Neural Heuristics
( NEAR) which treats program architecture search as an informed graph search problem.
The second method is DreamCoder, which aims to learn extensions of the DSL
(i.e., library learning). This part will conclude with a practical example of neurosymbolic learning for behavior analysis,
grounded in real-world applications for behavioral neuroscience.
|
12:00 to 13:00 |
Lunch
|
13:00 to 14:00 |
Talk: Jiajun Wu (Stanford)
(info)
Understanding the Visual World Through Naturally Supervised Code
Much of our visual world is inherently symbolic: scenes are made of objects; different objects may have the same color or material, with a grid layout; each object can be symmetric and have repetitive parts. How can we infer, represent, and use such symbolic structure from raw data, without hampering the expressiveness of neural networks? In this talk, I will demonstrate that symbolic structure, or code, can be learned from natural supervision. Such supervision can be from pixels, where neuro-symbolic methods automatically discover repetitive parts and objects for scene synthesis. It can be from objects, where humans during fabrication introduce priors that can be leveraged by machines to infer regular intrinsics such as texture and material. It can also be from language, which humans develop for the need of referring to and talking about objects, and where machines may learn grounded visual concepts only by looking at images and reading captions. When solving these problems, symbolic programs and neural nets play complementary roles: symbolic programs are more data-efficient to train and generalize better to new scenarios, as they robustly capture high-level structure; deep nets effectively extract complex, low-level features from cluttered and noisy visual and language data.
|
14:00 to 14:30 |
BREAK |
14:30 to 15:30 |
Tutorial 2b: Nadia Polikarpova (UCSD)/Shachar Itzhaky (Technion): Program Synthesis
(info)
Program Synthesis Tutorial Part 2
In this first part of the tutorial, students will get an introduction to symbolic program synthesis and will gain
some experience with existing synthesis tools.
|
15:30 to 15:45 |
Break |
15:45 to 16:45 |
Talk: Zenna Tavares (Columbia) |
16:45 to 18:00 |
Tutorial 3b: Vikash Mansinghka (MIT)
(info)
Probabilistic Programming Tutorial Part 2
|
18:00 to 20:00 |
Dinner |
|