Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking. We present an operational SGI definition grounded in the Practical Inquiry Model (PIM: Deliberation, Conception, Action, Perception) and operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning. SGI-Bench comprises over 1,000 expert-curated, cross-disciplinary samples inspired by Science's 125 Big Questions, enabling systematic evaluation of state-of-the-art LLMs. Results reveal gaps: low exact match (10--20%) in deep research despite step-level alignment; ideas lacking feasibility and detail; high code executability but low execution result accuracy in dry experiments; low sequence fidelity in wet protocols; and persistent multimodal comparative-reasoning challenges. We further introduce Test-Time Reinforcement Learning (TTRL), which optimizes retrieval-augmented novelty rewards at inference, enhancing hypothesis novelty without reference answer. Together, our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking.
- Year
- 2025
- Venue
- arXiv 2025
- Stars
- 166
- Authors
- 107
- Hosting
- Abstract onlyARXIV-DEFAULT
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/2512.16969ARXIV-DEFAULT
- TL;DR
- Semantic Scholar
Topics
2Abstract
Authors
107Bo LiuBowen ZhouYizhou WangXue YangWei LiBo ZhangLei BaiYang LiuChen TangHao WuJunchi YanXin ChenJunjun HeMing HuChenglong MaWanghan XuJiamin WuYingzhou LuYing ChenFengxiang WangYuanyuan ZhangXiangyu ZhaoJunzhi NingXinyao LiuYe DuChangkai JiCheng TangHuihui XuZiyan HuangJiyao LiuPengfei JiangYuchen RenBen FeiFenghua LingYuqiang LiQihao ZhengNanqing DongTianfan FuDongzhan ZhouYan LuWenlong ZhangWanli OuyangShixiang TangChunfeng SongSiqi SunXiangru TangHao KongZhen ZhaoJiaheng LiuHao ChenXiao-Ming WuXiangchao YanPeng YeShufei ZhangYuhao ZhouWenxuan HuangYifan ZhouSheng XuYixin ChenHuazhu FuXi ChenFeng LiuMao SuJiaqi WeiZhiqiang GaoWanhao LiuZaoyu ChenZijie GuoFangchen YuYuchen FuChenhui LiJingyi XuShuo LiZhiwang ZhouXuming HeJunchao GongXiang ZhuangQinglong CaoTianhao PengYuejin YangGuangshuai WangJia BuQiantai FengJiangbin ZhengYucheng WuFeifei JiangLihao SunChengbo LiJinzhe MaYating LiuKuo-Cheng WuShengdu ChaiOuwen ZhangjinWenbo CaoJunjie RenTaoyong CuiZhouheng YaoJuntao DengYijie SunWangxu WeiZhangrui LiZhiyu YaoXinyu GuRui SuWeikang SiLijing ChengJintai Lin