Ground Manipulator Primitive Tasks to Executable Actions Using Large Language Models

Authors

  • Yue Cao Purdue University
  • C. S. George Lee Purdue University

DOI:

https://doi.org/10.1609/aaaiss.v2i1.27720

Keywords:

Large Language Model, Language Grounding, Task Frame Formalism, Robot Architecture, Robot Control

Abstract

Layered architectures have been widely used in robot systems. The majority of them implement planning and execution functions in separate layers. However, there still lacks a straightforward way to transit high-level tasks in the planning layer to the low-level motor commands in the execution layer. In order to tackle this challenge, we propose a novel approach to ground the manipulator primitive tasks to robot low-level actions using large language models (LLMs). We designed a program-function-like prompt based on the task frame formalism. In this way, we enable LLMs to generate position/force set-points for hybrid control. Evaluations over several state-of-the-art LLMs are provided.

Downloads

Published

2024-01-22

Issue

Section

Unifying Representations for Robot Application Development