Skip to content
AI-generated research idea🟡Fairly easy🔧Minor TweakRelated work found

Semantic Contamination Detection for Benchmarks

Detect semantic contamination in benchmarks by calculating vector embedding similarities between evaluation items and training corpus samples.

Time

3-6 weeks on a ~1T-token proxy corpus, or 2-3 months at full frontier pre-training corpus scale

Compute level

Large cluster

Resources

Embedding and indexing a GPT-4-scale pre-training corpus (trillions of tokens across billions of documents) requires a multi-node GPU/CPU cluster and distributed vector storage; proxy validation on a ~1T-token open corpus requires 8-32 GPUs and multi-terabyte NVMe/RAM vector database infrastructure.

Prerequisites

basic ML knowledge, vector databases

Scope

large-scale data collection

The Gap

Appendix C describes the contamination check methodology, which relies on removing spaces/symbols and using substring matching. The authors admit this results in false negatives if there are small differences between evaluation and training data.

“Our substring match can result in false negatives (if there is a small difference between the evaluation and training data) as well as false positives.”
From the paper, Appendix C, page 29. Matched against the PDF text.

Limitation

The current substring-based contamination check fails to detect rephrased or semantically identical questions present in the training set, potentially inflating reported benchmark scores.

Proposed Solution

Implement a semantic contamination detection pipeline using vector embeddings (e.g., using a smaller encoder model) to calculate cosine similarity between benchmark questions and training corpus samples, identifying non-exact but semantically equivalent matches.

Why This Matters

Ensures the validity of claims regarding 'human-level performance' and prevents the community from optimizing on memorization rather than reasoning.

How to Get Started

How to test it: Test the hypothesis that dense vector embedding cosine similarity identifies paraphrased benchmark contamination missed by 50-character substring matching, using the paper's 3x50-character substring check as the baseline on standard benchmarks (e.g., MMLU, GSM-8K) against an open pre-training corpus augmented with synthetic paraphrases. Measure recall/precision on planted paraphrases and clean-vs-contaminated score delta, and stop if embedding retrieval yields less than 10 percentage points higher recall at a 5% false-positive rate or exceeds 20% false positives.

  1. Collect a sample benchmark dataset and a large pre-training text corpus subset.
  2. Generate dense vector embeddings for all benchmark questions and corpus chunks using an encoder model like Sentence-Transformers.
  3. Compute pairwise cosine similarities and establish thresholds to flag non-exact semantic matches.
  4. Compare the false-negative rate of the embedding approach against standard exact-match substring checks.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Works like TRACER: A Semantic-Aware Framework for Fine-Grained Contamination Detection in Code LLMs and Analysis of Semantic Benchmark Data Contamination Attack for LLM-Driven Fake News Detection explore semantic contamination detection techniques, but they leave open specific implementations using vector embeddings for general benchmarks as proposed. The listed works focus on specific domains or frameworks rather than a general encoder-based pipeline.

We searched OpenAlex for “Detect semantic contamination in benchmarks by calculating vector embedding similarities between evaluation items and training corpus samples.” and “Semantic Contamination Detection for Benchmarks” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

GPT-4 Technical Report

OpenAI · 2023

This report details the development of GPT-4, a large-scale multimodal Transformer model that exhibits human-level performance on various professional benchmarks, such as the Uniform Bar Exam. The authors emphasize the use of predictable scaling laws to estimate model performance from smaller compute budgets and employ Reinforcement Learning from Human Feedback (RLHF) alongside Rule-Based Reward Models (RBRMs) to align the model with safety guidelines.

Related Ideas

Is this a good idea?

Share onXLinkedIn