Skip to content
AI-generated research idea⛔Very hard🔬Significant ResearchNo close match found

Grounding AI Ideation: Multi-Objective RLCF for Feasible Scientific Discovery

A method that introduces a multi-objective comparison-based GRPO framework, combining scientific impact with a feasibility reward model trained on retracted and successfully reproduced papers.

Time

4-6 weeks for curating the feasibility preference dataset, training the auxiliary Feasibility Judge, running multi-objective GRPO on 4B/30B Thinker models, and conducting feasibility evaluations.

Compute level

Large cluster

Resources

64-128 H200-equivalent GPUs (8-16 nodes x 8 GPUs) with MS-SWIFT, DeepSpeed ZeRO-2/ZeRO-3, and vLLM to train the Feasibility Judge and run Comparison-Based GRPO (G=8, max completion 8,192 on 4,000 seed papers), plus curation of reproduced vs. retracted/failed paper pairs and domain-expert evaluation.

What the paper used

H200-equivalent GPUs with DeepSpeed ZeRO-2/ZeRO-3 and vLLM, scaled by model size: 32 GPUs (4 nodes x 8 GPUs) for 1.5B, 64 GPUs (8 nodes x 8 GPUs) for 3B-7B, and 128 GPUs (16 nodes x 8 GPUs) for 14B-32B models; trained via GRPO (batch size 128, 1 epoch, 8 generations/prompt) on 720,341 pairs (from 2.1M arXiv papers) for SciJudge and 4,000 seed papers for SciThinker.

Prerequisites

Reinforcement learning, multi-objective optimization, domain-specific scientific knowledge

Scope

new model architecture and build from scratch

The Gap

The ideation capability of Scientific Thinker is optimized solely for potential citation impact and evaluated by LLM-as-a-judge. The training objective lacks constraints on the physical, financial, or computational feasibility of the proposed experiments.

“Future work could explore broader formulations of scientific taste such as assessing the feasibility of experimental designs and better recognizing distinctive and diverse research ideas.”
From the paper, Section 6 (Limitations and Future Work), page 13. Matched against the PDF text.

Limitation

Generated ideas, while potentially high-impact in theory, may be experimentally unfeasible or practically impossible to execute in a real-world setting.

Proposed Solution

Implement a multi-objective Comparison-Based GRPO framework. Train an auxiliary reward model specifically on experimental feasibility using a dataset of retracted/failed papers versus successfully reproduced papers. Use a weighted combination of the 'Scientific Judge' (impact) and 'Feasibility Judge' (practicality) to steer the Scientific Thinker.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

Ensures that the AI-generated research proposals are not just theoretically popular but actually executable in real-world labs, bridging the gap between automated ideation and physical execution.

How to Get Started

How to test it: Test the hypothesis that multi-objective Comparison-Based GRPO combining SciJudge with a Feasibility Judge (trained on reproduced vs. retracted/failed papers) yields more executable ideas than single-objective SciThinker-4B/30B without sacrificing impact. Evaluate on the 200 in-domain and 200 out-of-domain 2025 seed papers using LLM-ensemble impact win rates and expert-graded or execution-based feasibility scores, stopping if impact win rate drops significantly without measurable feasibility gains.

  1. Curate a dataset of retracted or failed research papers versus successfully reproduced and implemented papers to capture feasibility signals.
  2. Train an auxiliary feasibility reward model using the curated dataset to classify experimental practicality.
  3. Integrate the feasibility reward alongside the existing scientific impact reward within a Comparison-Based GRPO framework.
  4. Benchmark the multi-objective ideation system against single-objective baselines for feasibility and novelty.

Code linked to this paper on Hugging Face: github.com/tongjingqi/AI-Can-Learn-Scientific-Taste

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 2 results

The listed papers discuss agentic browsing and adversarial paper-review dynamics, but none implement a multi-objective Comparison-Based GRPO framework using failed versus reproduced papers to balance impact and feasibility.

We searched OpenAlex for “A method that introduces a multi-objective comparison-based GRPO framework, combining scientific impact with a feasibility reward model trained on retracted and successfully reproduced papers.” and “Grounding AI Ideation: Multi-Objective RLCF for Feasible Scientific Discovery” and an AI compared the idea with the top 2 results. A small search can miss relevant work.

Source Paper

AI Can Learn Scientific Taste

Jingqi Tong, Mingzhe Li, Hangcheng Li +20 more · 2026

This paper explores whether AI models can develop 'scientific taste' to evaluate and propose high-impact research ideas. The authors propose Reinforcement Learning from Community Feedback (RLCF) to train models using large-scale citation data, successfully creating an AI that generates research proposals with higher potential impact than baseline models.

Related Ideas

Is this a good idea?

Share onXLinkedIn