Large Language Models

We study large language models from various perspectives such as programmability, controlled text generation, and different aspects of Trustworthy ML as part of our ongoing SafeAI project.

Publications

2024

Beyond Memorization: Violating Privacy Via Inference with Large Language Models

Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev

ICLR 2024 Spotlight, 2024 PPPM-Award

Paper
Code

Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation

Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev

ICLR 2024

Paper
Code

Watermark Stealing in Large Language Models

Nikola Jovanović, Robin Staab, Martin Vechev

Workshop on Reliable and Responsible Foundation Models (R2-FM) -- ICLR 2024 Oral

Paper
Code

Controlled Text Generation via Language Model Arithmetic

Jasper Dekoninck, Marc Fischer, Luca Beurer-Kellner, Martin Vechev

ICLR 2024 Spotlight

Paper
Code

Large Language Models are Advanced Anonymizers

Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev

arXiv 2024

Paper

Instruction Tuning for Secure Code Generation

Jingxuan He*, Mark Vero*, Gabriela Krasnopolska, Martin Vechev

arXiv 2024 * Equal contribution

Paper

Evading Data Contamination Detection for Language Models is (too) Easy

Jasper Dekoninck, Mark Niklas Müller, Maximilian Baader, Marc Fischer, Martin Vechev

arXiv 2024

Paper
Code

Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation

Luca Beurer-Kellner, Marc Fischer, Martin Vechev

arXiv 2024

Paper

2023

Large Language Models for Code: Security Hardening and Adversarial Testing

Jingxuan He, Martin Vechev

ACM CCS 2023 Distinguished Paper Award

Slides
Paper
Code

Prompt Sketching for Large Language Models

Luca Beurer-Kellner, Mark Niklas Müller, Marc Fischer, Martin Vechev

arXiv 2023

Paper

LMQL Chat: Scripted Chatbot Development

Luca Beurer-Kellner*, Marc Fischer*, Martin Vechev

Neural Conversational AI Workshop, TEACH -- ICML 2023 * Equal contribution

Paper
Code

Large Language Models are Zero-Shot Multi-Tool Users

Luca Beurer-Kellner*, Marc Fischer*, Martin Vechev

Knowlege and Logical Reasoning Workshop -- ICML 2023 * Equal contribution

Paper
Code

Prompting Is Programming: A Query Language for Large Language Models

Luca Beurer-Kellner, Marc Fischer, Martin Vechev

PLDI 2023

Paper
Code