Publications

Accepted papers

  • SuperSycophantic: Stress-Testing Frontier Models from Single- to Multi-Turn Sycophancy

    T. Zhang, O. Yasunaga, W. Jiang, J. Bo, F. Draye, A. Javadov, Arth Singh, L. Muir, V. Oldemburgo De Mello, B. Schölkopf, Z. Jin

    NeurIPS 2026 Paper Code

    Stress-tested 9 frontier models on objective and subjective sycophancy under first-turn framing and multi-turn pressure. GPT-5.4 changes up to 37.3% of its correct objective answers and switches in up to 97.5% of subjective trials; telling models to recheck from scratch before replying cuts sycophantic switching by more than half.

  • Predicting the Critic: Training on User Feedback Reshapes Emergent Misalignment

    X. J. Zhou, Arth Singh, U. S. T. Carr, E. Tennant

    AI4Good @ NeurIPS 2026 Oral 🎉 and DevAI @ NeurIPS 2026

    Appends a one-line user reaction to 7,049 bad-medical-advice fine-tuning episodes and varies only whether the reaction tokens count toward the loss. Trained on, criticism teaches the model to predict its critic and can raise emergent misalignment (reliably on Qwen2.5); placed before the question as a note, it lowers misalignment in all five model families by ten points on average, but restoring the note-format context at evaluation brings it back.

  • Fluid Reasoning Representations

    D. Kharlapenko, T. Zhang, Arth Singh, A. Stolfo, A. Conmy, M. Sachan, Z. Jin

    Findings of EMNLP 2026 arXiv

    Introduces FRRs, a representation-level account of how extended-thinking LLMs organize action and predicate concepts. On obfuscated planning, symbolic, and math tasks, cross-naming steering and causal patching show reasoning sharpens a dynamic already latent in base models.

  • Mechanistically Interpreting Compression in Vision-Language Models

    V. Elluru, Arth Singh, R. Aguero, A. Agarwal, D. Das, H. Paul

    Mechanistic Interpretability Workshop @ ICML 2026 arXiv

    Circuit and crosscoder analysis of how pruning and quantization change VLM internals, plus VLMSafe-420, a harmful-vs-benign-counterfactual benchmark showing pruning sharply erodes genuine refusal.

Preprints

  • Paper Tigers: Jailbreak Judges Mistake Fake Answers for Real Harm

    Arth Singh*, S. Simko*, T. Zhang, B. Schölkopf, Z. Jin (* equal contribution)

    Preprint, November 2026 Project page Code

    Benchmarked 21 jailbreak judges on 2,649 answers that include fake ones: the HarmBench classifier calls 81% of fake answers successful jailbreaks, and 14 of the 20 baseline judges misstate attack success rates by 20+ points. Released the benchmark and Pugmark, a 27B LoRA judge fooled by only 2.8% of fakes (score 88.7 vs. 75.7 for the next best judge).

  • Mechanistic Analysis of Alignment Algorithms in Language Models

    A. Sinha, I. Garg, V. Elluru, Arth Singh, K. Garg

    arXiv preprint, May 2026 arXiv

    A mechanistic comparison of six preference-optimization methods (PPO, DPO, SimPO, ORPO, GRPO, and KTO) across three open-weight model families using linear probes, sparse autoencoders, and crosscoders. KTO and GRPO sharpen the linear separability of preference signals, DPO and ORPO degrade it through geometric rotation and feature attenuation, and PPO and SimPO largely preserve the base model's geometry.

  • Toward Human-AI Complementarity Across Diverse Tasks

    Y. Xu, A. Dahmani, M. D. Blanchard, N. Dern, E. Nastase, F. Bianco, M. Pavlovic, S. Krishna, E. Modesitt, M. A. Christ, Arth Singh, G. Molinaro, S. B. Sengupta, J. Pamarthi, A. Menon, R. Jain

    arXiv preprint, March 2026 arXiv

    Investigated human-AI complementarity through hybridization and two AI assistance methods on a 1,886-sample multi-domain dataset. Led the analyses for quantitative overreliance, GPQA positional bias, and oracle hybridization bounds.