Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models

Junda Zhao, Yuliang Song, Eldan Cohen. Journal of Systems and Software, 2025. Presented in the Journal-First track of ASE 2025, Seoul, South Korea.

Pre-trained language models of code generate a single summary for a code snippet, although many valid summaries usually exist. Variational Prefix Tuning (VPT) integrates a conditional variational autoencoder (CVAE) framework into existing pre-trained transformer models as a modular component. Sampled latent codes are injected as continuous prefixes during decoding, so that the model can produce multiple diverse yet accurate summaries without costly retraining of the backbone. A bi-criteria reranking method then selects the set of summaries that best balances diversity and accuracy.

• Paper (open access)
• Code
• Slides and poster from ASE 2025