Skip to content

Source paper

Self-Supervised Prompt Optimization

Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu and 6 more

Jinyu Xiang, Jiayi Zhang, Zhaoyang Yu, Fengwei Teng, Jinhao Tu, Xinbing Liang, Sirui Hong, Chenglin Wu, Yuyu Luo

arXiv 2502.06855 · 20252 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

The paper introduces Self-Supervised Prompt Optimization (SPO), a framework designed to automate the improvement of prompts for Large Language Models (LLMs) without relying on ground truth data or human feedback. SPO leverages the inherent capability of LLMs to distinguish better answers through pairwise comparisons (Output vs. Output) to guide the optimization process, achieving state-of-the-art performance at a fraction of the cost of existing methods.

How it works (AI summary)

SPO operates on an iterative 'Optimize-Execute-Evaluate' loop. It starts with an initial prompt and a small set of unlabelled questions. In each iteration, an LLM optimizer generates a new prompt variant. This prompt is executed to generate outputs. Crucially, instead of using ground truth, an 'LLM-as-a-judge' performs pairwise comparisons between the new outputs and the previous best outputs to determine the winner. This self-generated signal guides the next round of prompt refinement.