% This text will be sanitized and placed into lqa-output/abstract.txt We introduce LMQL Actions, a framework and programming environment to facilitate the implementation of tool-augmented language models (LMs). Concretely, we augment LMs with the ability to call actions (arbitrary Python functions), and experiment with different ways of tool discovery and invocation. We find that, while previous works heavily rely on few-shot prompting to teach tool use, a zero-shot, instruction-only approach is enough to achieve competitive performance. At the same time, LMQL Actions zero-shot approach also offers a much simpler programming interface, not requiring any involved demonstrations. Building on this, we show how LMQL Actions enables LLMs to automatically discover and combine multiple tools to solve complex tasks. Overall, we find that inline tool use as enabled by LMQL Actions, outperforms existing tool augmentation approaches, both in arithmetic reasoning tasks and text-based question answering.
Large Language Models are Zero-Shot Multi-Tool Users
Luca Beurer-Kellner*, Marc Fischer*, Martin Vechev
Knowlege and Logical Reasoning Workshop -- ICML 2023* Equal contribution
@misc{BeuererkellnerFV2023, title = {Large Language Models are Zero-Shot Multi-Tool Users}, venue = {ICML Workshop on Knowledge and Logical Reasoning in the Era of Data-Driven Learning}, year={2023}, author = {Luca Beurer-Kellner and Marc Fischer and Martin Vechev}, }