Creating Augmented Reality Applications Using Large Language Models: Experiments with the CMAG Framework and ARWFMM

Abstract

In recent years, there has been a notable advancement in machine learning technology, resulting in the creation of commercial and open-source large language models (LLMs) such as ChatGPT and Llama. These models are currently being investigated for their potential applications in various fields, including the generation of code for augmented reality applications. Given the complexity of augmented reality, the direct generation of code for such applications using LLMs remains complex and hardly verifiable. Therefore, we examine in this paper how the previously introduced concept of Conceptual Model Augmented Generative Artificial Intelligence (CMAG) can support the comprehensibility of an LLM's output on the basis of the Augmented Reality Workflow Modeling Method (ARWFMM). We illustrate the results from a first experiment, which are promising for future work in this area.