Language Grounding for Robotics Workshop at ACL 2017
Vancouver, Canada
Dates: Aug 3 or 4, 2017 (TBD)


The embodied, task-oriented aspect of language grounding is an important and timely next research direction following the exciting recent progress in visual language grounding. To realize the long-term goal of robots that we can converse with in our homes, offices, hospitals, and warehouses, it is essential that we develop new techniques for linking language to action in the real world. Can we give instructions to robotic agents to assist with navigation and manipulation tasks in remote settings? Can we talk to robots about the surrounding visual world, and help them interactively learn the language needed to finish a task? We hope to learn about (and begin to answer) these questions in the proposed workshop on Language Grounding for Robotics.

This workshop aims to bring together members of the NLP, robotics, and vision communities to focus on the much-needed task-oriented aspect of language grounding. The confirmed invited speakers, program committee, and organizing committee (see below) consist of researchers who belong to these communities, and who work at the intersection of language, robotics, and vision.

Topics of interest for this workshop include, but are not limited to:
  • Aligning and Translating Language to Situated Actions
  • Simulated and Real World Situations
  • Instructions for Navigation
  • Instructions for Articulation
  • Instructions for Manipulation
  • Skill Learning via Interactive Dialogue
  • Language Learning via Grounded Dialogue
  • Language Generation for Embodied Tasks
  • Grounded Knowledge Representations
  • Mapping Language and World
  • Grounded Reinforcement Learning
  • Language-based Game Playing for Grounding
  • Structured and Deep Learning Models for Embodied Language
  • New Datasets for Embodied Language
  • Better Evaluation Metrics for Embodied Language
Format: The proposed format is 6-7 invited speaker talks, several oral paper talks, poster sessions, and a concluding panel with 4-5 of the invited speakers to discuss the concrete future directions, challenges, and collaboration opportunities for this important topic.

Confirmed Invited Speakers:

Regina Barzilay, MIT
Joyce Chai, MSU
Dieter Fox, Univ of Washington
Raia Hadsell, Google DeepMind
Hadas Kress-Gazit, Cornell
Percy Liang, Stanford
Ray Mooney, UT Austin
Nicholas Roy, MIT
Stefanie Tellex, Brown
Jason Weston, Facebook AI Research

Steering Committee:

Julia Hockenmaier, UIUC
Ray Mooney, UT Austin
Luke Zettlemoyer, Univ of Washington


Mohit Bansal, UNC Chapel Hill
Cynthia Matuszek, UMBC
Jacob Andreas, UC Berkeley
Yoav Artzi, Cornell
Yonatan Bisk, ISI, USC

Confirmed Programme Committee:

Antoine Bordes, Facebook AI Research
Ankur Parikh, Google Research
Devi Parikh, Georgia Tech
Dhruv Batra, Georgia Tech
Dieter Fox, Univ of Washington
Dipendra Misra, Cornell
Edward Grefenstette, Google DeepMind
Eunsol Choi, Univ of Washington
Hadas Kress-Gazit, Cornell
Hannaneh Hajishirzi, Univ of Washington
Hongyuan Mei, JHU
Jason Weston, Facebook AI Research
Jayant Krishnamurthy, AI2
Jesse Dodge, CMU
Jesse Thomason, UT Austin
Jivko Sinapov, UT Austin
Jonathan Berant, Tel-Aviv
Joyce Chai, MSU
Julia Hockenmaier, UIUC
Karthik Narasimhan, MIT
Kenton Lee, Univ of Washington
Licheng Yu, UNC Chapel Hill
Lisa A Hendricks, UC Berkeley
Lucy Vanderwende, MSR
Marcus Rohrbach, UC Berkeley
Mark Yatskar, Univ of Washington
Matthew Walter, TTI-Chicago
Raia Hadsell, Google DeepMind
Ray Mooney, UT Austin
Siva Reddy, Univ of Edinburgh
Subhashini Venugopalan, UT Austin
Tom Kwiatkowski, Google Research
Tom Williams, Tufts
Yejin Choi, Univ of Washington

Sponsorship and Funding:

We are also in touch with potential sponsors for the workshop and they have expressed initial interest in funding. We hope to arrange both a best paper award and a few student travel scholarships to encourage members of the robotics community to attend.