孙圣尧 (Sun Shengyao)

孙圣尧 (Sun Shengyao)

he/his

Incoming second-year undergraduate at SJTU SPEIT · LLM & software-agent evaluation · interested in embodied intelligence & robotics

Publications

Preprints | 预印本

FrameShift-CAD: Executable Coordinate-Frame Interventions for Diagnosing Text-to-CAD Generation

Published in Research Square (preprint), 2026

A controlled diagnostic study of text-to-CAD systems, using matched active-object and passive-frame prompts with programmatically constructed counterfactuals to reveal an operation-specific translation–rotation reliability gap.

Recommended citation: Sun, S. (2026). "FrameShift-CAD: Executable Coordinate-Frame Interventions for Diagnosing Text-to-CAD Generation." Research Square. doi:10.21203/rs.3.rs-10904095/v1
Download Paper

Did It Happen? Counterfactual Evaluation of LLM Agent Recovery from Ambiguous Tool Outcomes

Published in Research Square (preprint), 2026

A counterfactual evaluation framework studying whether LLM agents can detect, clarify, and recover from ambiguous tool outcomes instead of proceeding on wrong assumptions.

Recommended citation: Sun, S. (2026). "Did It Happen? Counterfactual Evaluation of LLM Agent Recovery from Ambiguous Tool Outcomes." Research Square. doi:10.21203/rs.3.rs-10730245/v1
Download Paper

Mutation Testing of Task-Scoped State Oracles in Software-Agent Benchmarks: A Cross-Benchmark Empirical Study

Published in Research Square (preprint), 2026

A cross-benchmark empirical study applying mutation testing to task-scoped state oracles to calibrate the reliability of software-agent benchmarks.

Recommended citation: Sun, S. (2026). "Mutation Testing of Task-Scoped State Oracles in Software-Agent Benchmarks: A Cross-Benchmark Empirical Study." Research Square. doi:10.21203/rs.3.rs-10665114/v1
Download Paper

Testing JSON Schema Instruction Artifacts: Distributional Robustness under Validation-Equivalent Serialization and JSON Mode

Published in Research Square (preprint), 2026

An empirical study of how validation-equivalent JSON Schema serialization order affects black-box LLM generation, with practical guidelines for schema design.

Recommended citation: Sun, S. (2026). "Testing JSON Schema Instruction Artifacts: Distributional Robustness under Validation-Equivalent Serialization and JSON Mode." Research Square. doi:10.21203/rs.3.rs-10610100/v1
Download Paper